How I executed my first paid task
I will get to the main details in the very beginning. I got paid about 800usd for this project and it took me 2–3 weeks to get it done. You guys can interact with the project here. Just remember, The company name is the company ticker(In upper case). So for example Apple’s ticker is AAPL. So type in AAPL and the year (2018,2019). Thank you
Now, Let’s begin.
The client approached me with a “simple” task. Get financial data for over 3500 companies listed on the NYSE and on NASDAQ for the past 20 years, clean it up to remove any discrepancies, and represent it using a dynamic front end.
Sounded simple enough, the only problem being, “Where do I get that much data from?”
- Getting the data
After some research and intense “googling”, I came across Financial Modelling Prep.
A hub for all data finance-related, FMP has everything a finance enthusiast would need. From Financial Ratio on the quarterly and annual level to cryptocurrency historical and real-time data to educational content. Those guys have done quite the job, I must say.
Using one of their APIs, I was able to fetch data for any company of my choice and for any year range. So yes, FMP really solved the toughest part of the project f which was getting the right data for the right time period.
- Cleaning the data
As there was quite the data being gathered, It’s obvious there would be erroneous values due to bad calculations or human error.
I tried to resolve as many bad values as I possibly could myself but eventually, It was too much of a task and I got in touch with FMP’s
creator and brought this issue up with him.
They worked with me to resolve as many erroneous values as I possibly could find but in the end, I realized,
This would be too much of a time-intensive task that would require more resources too. I did propose looking at other sources for information such as Alpha Vantage or IEX cloud but there was no guarantee from their end either that their data was error-proof.
The client agreed with me on this point and gave me the go-ahead to carry on with the project.
- Bringing it all together
Probably my favorite part of the entire project.
After all the data was collected from FMP, we decided to move to the backend creation of the project on Django.
Different views were created which upon request would respond with the information needed.
Being someone comfortable with Python, I did not have trouble writing the backend code in Django but I did need some help creating the dynamic front end and that’s where I got some help.
I got a friend of mine to create an interactive front end on Angular which would communicate with the Django backend and display the appropriate data.
After completion of both the front and backend, We deployed the project onto a Digital Ocean Server and handed it over. The client was quite pleased with the pace of the project.
The most satisfying part of the entire project(apart from receiving the money) was the fact that I utilized my skill set to create something meaningful and valuable.
The ability to create some productivity really gives you that satisfactory feeling of accomplishment that can’t be gained otherwise.
The road to becoming a Data Scientist is hard and arduous but so is everything else in life which is worthwhile.
The journey continues…