Web scraper and data processing job for retail websites

Experienced web scraper and data processor to develop daily scraping service off 500+ retail product websites and one single affiliate network. Please only apply if you have referenceable web scraping and automation processing experience at this scale. Please explain the relevance of your experience within your application.


To scrape products from 500+ retailer UK websites, all of which have 1,000’s of products (including colour/size/style variants). The data to be scraped is retail product data on the retailer's product detail page, including all available images (at full resolution). Example: [login to view URL]

To scrape commission details for the same retailers (advertisers) from a single affiliate network.

These products need to be scraped daily and then:

• Stripped down to a subset of products based on a simple, single identification criteria (details provided to successful application). E.g. 5,000 products from a retail website might be reduced to 200 to be saved. (it might be that this reduction can be done before the data is scraped)

• Mapped to a provided predefined category taxonomy – e.g. John Lewis might have Mens / Clothing / Tops / Polo Shirts and this will be mapped to Mens / Clothing / Shirts / Polo Shirts

• Appended with the appropriate tracking URL elements for a single affiliate network which will be confirmed to successful applicant (e.g. Affiliate Window, Commission Junction)

• Enriched with Shopify product tags based on data within the products title, description, colour, brand etc

• Enriched products with data from the a single affiliate network which will be confirmed to successful applicant (commission % etc.)

• Consolidated in to a single file/repository

• Enriched with a calculation of the product price / affiliate commissions % (e.g. £30 RRP / 5% Commission = £1.50)

• Uploaded to a Shopify website

The successful applicant will setup the required services and database/repositories on an Amazon AWS machine.

I have had a few conversations with freelancers that have submitted which indicate that my brief is not clear. For the avoidance of doubt:
- This project is to deliver a stand alone solution that will scrape 500 websites every day
- The products to be extracted will likely be 10,000+ (once a majority of the products on websites have been disregarded).
- This process is not a one off, it will be run every day
- The solution will not involve your team performing tasks every day. It needs to be an autonomous solution that requires no further involvement from the developer.
- I will provide examples of data to be scraped and list of retailers to any freelancers that can demonstrate that they have relevant experience will complex, ongoing web scraping like this.
Thanks :)

Habilidades: Marketing de afiliados, Procesamiento de datos, Shopify, Extracción de datos web

Ver más: looking for an experienced web scraper, data processing job request, data processing job paypal, projects on web scraping, web scraping salary, data scraping jobs, online web scraping jobs, data scraping tutorial, how to do data scraping, python web scraping jobs, freelance data scraping, data processing, web scraping, shopify, data processing job online job yahoo answer, data processing job german deutsche post type data, cost involved data processing job online home, cool web applications data processing, captcha data processing job, data processing job sharjah

Información del empleador:
( 1 comentario ) Leamington Spa, United Kingdom

Nº del proyecto: #15128808

39 freelancers están ofertando el promedio de $1387 para este trabajo


Hi, my name is Sevinc. I read your "Web scraper and data processing job for retail websites" project descriptions carefully before bidding. I checked your requirements as well... I got what you need and ready to go a Más

$6000 USD en 45 días
(283 comentarios)

I can extract the products from 500+ UK retailer website. Relevant Skills and Experience Affiliate Marketing, Data Processing, Shopify, Web Scraping Proposed Milestones $1250 USD - Web scraper and data processing job Más

$1250 USD en 10 días
(179 comentarios)

Hello Sir/Madam. I'm an experienced web data extractor and I'd like to have a chat with you to refine and define some points. Habilidades y experiencia relevante Experience building solutions like: Windows Desktop App Más

$1200 USD en 10 días
(196 comentarios)

Dear Sir/Ma'am, I am a Web research, Data Entry & Web Scrapping expert. I checked and understood your requirements. Relevant Skills and Experience Data Processing, Shopify, Web Scraping Proposed Milestones $1500 Más

$1500 USD en 20 días
(235 comentarios)

Hello sir, I am SHOPIFY EXPERT ! Store setup, theme modifications,custom design and modify/create store functionality. Relevant Skills and Experience Some Latest Shopify Ecommerce Projects:- [login to view URL] Más

$1176 USD en 20 días
(109 comentarios)
$750 USD en 14 días
(240 comentarios)

Nice to meet you. I 've read your post and so very interested in your job. I 'm sure I can satisfy you with my ability. I 'm working in part of coding and software developing for over 6 years. I have experiences th Más

$1250 USD en 20 días
(62 comentarios)

I am an expert in delivering custom scripts and projects Relevant Skills and Experience proven track record on this site Proposed Milestones $1058 USD - milestone

$1058 USD en 20 días
(67 comentarios)

i can do this task since i am having experience on it. Already done same kind of scrapping project( product name, image, price etc..) Relevant Skills and Experience web scrapping Proposed Milestones $1250 USD - Más

$750 USD en 20 días
(11 comentarios)

Hi Future Client, Our business is to make your business effective, profitable, & [login to view URL] are able to have a sound understanding of our Clients’ operations and their corporate goals, and we continually bring kno Más

$1500 USD en 20 días
(20 comentarios)
$1000 USD en 20 días
(124 comentarios)
$1250 USD en 20 días
(3 comentarios)

Hi I am an expierienced python developer specializing in web scraping. I have written scrapers for over 100 websites. Including: Yelp, Grubhub, Foodora, H&M and much more. My relevant skills are Python+ various scrapin Más

$1098 USD en 5 días
(72 comentarios)

Dear Hiring Manager, You may see my profile here: https://www.freelancer.com/u/motiur03.html Relevant Skills and Experience Gumtree, Craiglist Advert posting, web search, web scrapping, Amazon, eBay, Shopify, WooComm Más

$750 USD en 5 días
(120 comentarios)

Hello, how are you? Here is the best scrapper you are looking for. I've just read your project description and i am interested in it Relevant Skills and Experience I have full experiences in web scrapping and i've alr Más

$1323 USD en 20 días
(15 comentarios)

Build a tool which will be automatically scrape data from 500+ websites, massage it and upload it on shopify platform Relevant Skills and Experience web scarping/ecommerce Proposed Milestones $5000 USD - Completed to Más

$5000 USD en 60 días
(47 comentarios)

Hello, I willl do this scraping daily. I have seen your website and I can scrap 500 product every day in your time. I can scrap almost every website. Relevant Skills and Experience I have best scrapping skill and I ca Más

$750 USD en 5 días
(79 comentarios)

Hello, I cantake up this task and will ensure the task is delivered to your level of satisfaction. I have sound experience in data scarpping through script and have done similar project of large scale Relevant Skills Más

$1000 USD en 20 días
(59 comentarios)

Hello Please check my recent developed scrappers. [login to view URL] [login to view URL] Best Regards Relevant Skills and Experience https://prnt Más

$1352 USD en 21 días
(34 comentarios)

Hello! I have read all the requirements carefully and I have all the required skills. I am extremely interested in this project and would love to work with you on long term basis, kindly reply back to discuss further r Más

$1323 USD en 20 días
(14 comentarios)