Note: The first three payment/milestones below fall under “Applying text algorithm again, part1”, and the second three under “Applying text algorithm again, part2”
The aim of this project is to improve a classification+clustering method developed in a previous project, based on some technical issues that were identified.
1. The team is going to send the freelancer a list of irrelevant words.
2. As agreed in the end of the previous project, the same overall method should be carried out, with the same tools. As before, we are going to try three different levels of tolerance for the algorithm. We are going to try five versions of algorithm+clusterization to deal with the irrelevant word problem:
i. considering only words which occur in less than 10% of entries to create the algorithm and clustering
ii. considering only words which occur in less than XX% of entries (another cut-off chosen after the results from iii.) to create the algorithm and clustering
iii. ignoring words from the list of irrelevant words #1 to create the algorithm and clustering
iv. ignoring words from the list of irrelevant words #2 to create the algorithm and clustering
v. a version combining the use of the best list of irrelevant words and the best frequency cut-off
(In all versions we use the tool to fix typos. For each version we test three tolerance levels.)
3. In each case, compute the silhouette score both on predicted codes and on clusters
4. The team will assess the results using the list of irrelevant words #1 and if necessary bring some modifications to the list for the algorithm and clustering to be re-run (version ii. using list of irrelevant words #2).
5. Once the algorithm and clustering are finalized: assign a predicted code to each cluster, by comparing of "mean cluster sentence" with all code descriptions (from initial learning dataset + additional codebook) to choose the best matching code description.
6. The project ends when the algorithm and the clustering perform in a satisfactory way.
The team will then receive from the freelancer the codes/tool allowing them to re-run the exact same algorithm and clustering in the future and adjust them if necessary.
Payment plan:
First $278 for 2i, including algorithm results, clusterization results, silhouette score, and predicted code assigned to each cluster
Second Same, for 2ii
Third Same, for 2iii
Fourth Same, for 2iv
Fifth Same, for 2v
Sixth $280 for all codes/tools necessary to re-run and adjust the algorithm, clusterization, silhouette score calculation, and code for assigning codes to clusters
Masters of science & professional statistician is here to help.................................................................................................................................
Hi I am a very experienced statistician and academic writer. I have completed several PhD level thesis projects involving advanced statistical analysis of data. I have worked with data from several companies and have done projects involving high level quantitative analysis and data interpretation skills to study the trends, time behaviour and compare the variables in the data. I can do advanced level analysis in SPSS, R, WEKA and excel tools like machine learning, hypothesis testing, forecasting, T-test, ANOVA etc.
Looking forward to discussion,
Best Regards,
Suyash
note: I have previous experience in similar works
hi,
basically I'm an electronics engineer. expert in python. Surely I could help you.
Come to chat for more discussion.
thank you
Data Analyst/Scientist with more than 6+ years of experience in R Language, SPSS, STATA, SAS, MINITAB. I have been doing descriptive and inferential statistics
Key Techniques are
Regression Model
Binary Logistic Model
Factor Analysis
Cluster Analysis
Neural Network
Parametric and Non-Parametric Test
Good in data visualization using Data mining technique like CRT,QUESTetc
Please refer my client's feedback ( 5 star rated) Kindly reach out to me for further discussions