Gathering data

After deciding what is the initial approach, the next step is to stat gathering the data that will allow you to build the data warehouse.

As we discussed, the data can be:
  • End of the day prices
  • Dividends
  • Splits
  • Macro-economic data
  • Micro-economic data
  • Weather
  • Public holidays
  • Important events
This is a simple but tricky step. The reason is that since I am an amateur investor with not much appetite to spend money on gathering data, I need to use one of the free services which are available.

EOD/Dividends/Splits
Until recently, Yahoo Finance and Google were the most popular choices for such a task. Unfortunately, after the acquisition of Yahoo from Verizon, Yahoo discontinued the free service of bulking downloading data and you can only download one file at the time.

So after May 2017, the available options are Google and an Excel macro or Yahoo and a script which will  replicate the crumb/cookie mechanism.

 Public holidays
Any weather site

Popular posts from this blog

Building the model

The last few years, I have ended in the following 7 or 8 steps before investing real money in the market. Create the approach and the hypothesis Select your tools & build your data warehouse Quantify the approach Finalize the weights of the criteria Define the trading rule Back-Test Is it profitable? Trade Step 1.  Create the approach and the hypothesis Identify the statistical tools and financial theories that will allow you to build a profitable model. For example, if the value of the Bollinger Bands is above or below X and the Organic growth of the Company is above Y percent then shortlist the stock for investment.   Step 2. Select your tools & build your data warehouse In order to have a working model you need to have data for a long period of data. This will allow you to test the model before you start investing and will allow you have efficient predictions. The idea is to have data in a form of time series in order to allow you to compare...
Read more

Building the data warehouse

This step is the reason of starting the 3rd trip in trading and not continuing the previous approach. The target is to run calculations faster with a bigger amount of data and be able to access the data from any available laptop around the world. The idea is that instead of spending time to work on an Excel based platform, I will build all the calculations on a SQL platform and I will just use Excel or a web-based solution (e.g. PowerBI)  to report the results. In 2017, cloud computing has been evolved massively and it is really easy to get an SQL database from one of the big Software as a Service cloud (SaaS) vendors. Microsoft Azure and Amazon Web services are the two vendors that I am most familiar with and I decided to compare them and use one of the two. The criteria are functionality, cost and reliability. Reliability: Same level for both vendors Functionality: +1 for Amazon since they offer the on premise version of the SQL server while Azure offers a cloud vers...
Read more

The influence

Before moving forward, I would like to remind myself the 4  professors that mostly influenced me towards this approach: Antonios X Panagiotopoulos Christos Agiakloglou Ioannis Pollalis Christos Alexakis   Each one of them gave me one or more "lessons" during the 4 years of my BSc studies. Antonios X Panagiotopoulos He was one of the greatest professors that I have met in my life yet. He was Head of Professors at the Department of Informatics and he was teaching two optional courses in the BSc in Economics at University of Piraeus and I think that at the beginning he considered my as one of his worst students. This professor changed my life by introducing us to the theories of Mathematical Programming and Mathematical Logic. He was a unique professor. One of the days that will be in my mind for ever is the day that in front of the whole amphitheater, he gave me a lecture because when I asked a question, I was disrespectful to Mathematics. Since that day, I ...
Read more