Dynamic Webscraper for Financial Website (probably uses AJAX)

[Hi I originally posted project asking for solution in Python code, but a number of people asked me about other languages, so I have reposted!] I have written a financial statistics app in Python and I need a webscrapper to provide data to it. You can use another language you want for your webscrapper, [Ruby, PHP etc] as long as: **1)** I can run it on my low spec Linux machine [Please no .NET framework, so no C# etc] **2)** Your webscrapper can communicate with my Python program, eg ideally by passing a value via a method call etc [a mySQL database could be used but I think slow and heavy, a method call would be much lighter and faster!] _BACKGROUND_ I am interested in derived financial markets for a research document I'm writing. I need to gather real time information on various financial markets and run statistical comparisons on data generated. I will use IGIndex platform for webscraping, ***the problem is data changes every second!! Are YOU clever enough to webscrape this website?!*** _HOW I'LL RUN THIS PROJECT_ Right now, this project is just for 4 small markets for FTSE. 3 are show on the video, the 4th is just the FTSE index, also on the same website (I can provide details later:) p.s. Please don't worry if you don't know what FTSE is, you don't need to know this, I just have some data that needs to be webscrapped :) _WHAT I WANT_ I have made a short video 2.5 minute containing screenshots of what I'm after, please watch this before continuing. Thank-you! [login to view URL] [Note the video is only 2.5mins however the upload caused an error and it last 7.5 mins, there is 5mins of black screen at end!] ## Deliverables _DETAILS:_ Log in to IGIndex [you can create a free account] Navigate to: (see video) 1 Binary Indices -> FTSE (Daily) -> FTSE to rise + FTSE to fall 2 Binary Hourlies -> Indicies -> FTSE -> (Hourly) -> FTSE to rise + FTSE to fall 3 Binary (20 min Markets) -> FTSE -> FTSE to rise + FTSE to fall Very important: This value continually updates and sometimes changes quickly, eg sometimes 100 times in a minute.. I need to know immediately when this value has changed because I'm running my tests with a highly accurate digital clock and any lag delays will mess up the experiment. _DELIVERABLES:_ * Fully working source code + clearly commented application. * Logs in to IGIndex (password and log in should be in the code so I can change it for mine later, simply create a free account to test.) * The application continually scrapes data multiple times a second for various FTSE markets and has the following features: * Scrapes 'FTSE to finish UP' and 'FTSE to Finish Down' for markets above. I'd like to know the: SELL price BUY price The Update time The reference price the bet is referring to eg 'FTSE to finish Up [5629.09]' the reference value is 5629.09 in this case. * The market time needs to be displayed, eg for 20 min FTSE time could be e.g. [10.20am to 10.40am] * This form needs to update **immediately** the website updates eg within 100ms, it needs to be low lag time. * The data scraped needs to be available in some kind of structure that can then be accessed by my Python application quickly and easily. _FUTURE:_ If successful, I will also want to get data of several other website for different markets and expand to other brokers to compare data, if you do a good job may be more work on offer. Thank-you Kindly for taking time to watch video and read this.