Find Jobs
Hire Freelancers

Data Pipeline json in Blob to Sql

$30-250 AUD

Cerrado
Publicado hace alrededor de 7 años

$30-250 AUD

Pagado a la entrega
Afternoon, I would like to create a data pipeline using Azure Data Factory between json files which appear in my Azure Blob containers and my Azure Sql Tables. My blob currently has 3 containers and receives dates from a web app from API pulls from three platforms, Square, Deputy and Xero. The data in each container is subtly different but all in json. Part One: I would like you to create a data pipeline for the data in the square container which appends to an Sql Table you create. There should be 3 pipelines and 3 tables. 1. Historical Data and Live Data is placed into the same sql table and simply appended. 2. Modifier List is placed into it’s own sql table. A new modifier list file will appear every week and it should replace the data in the it’s sql table. It will always have the same schema & columns, it might just have a different number of rows and values. 3. Item List is placed into it’s own sql table. A new item list file will appear every week and it should replace the data in the it’s sql table. It will always have the same schema & columns, it might just have a different number of rows and values. Question: Are you able to achieve this with the data as they are represented in the containers at the moment or would it be easier for each pipeline to have it’s own container store? The Historical data files name will always begin with “Historical_Data_”. The modifier list will always start will always contain “_ModifierLists_” in the name. The Item List will always contain “_Items_” in the name. Part Two: I would like you to create a data pipeline for the data in the xero container. The data will flow into the container every week, it will contain the same schema but maybe different values and these should just be appended to the sql table you create. Part Three: I would like you to create a data pipeline for the data in the deputy container. The data will flow into the container every few days, it will contain the same schema but maybe different values and these should just be appended to the sql table you create. I have already attempted to create the pipeline myself but I’m unable to expand the nested arrays in a manner as they work completely. The Xero Component (Part Two) will most likely be the most time intensive as there are quite a few nested arrays, but all of which follow the same format. I’ve included my sql query code for creating a table for the Square Historical and Live Data if you’d like to use that to save you some time in writing the sql code. Part One Files (Square): Historical Data: Name: Historical_Data_Neoscona_2017-02-18 23:27:39txt Download Link: [login to view URL]!At8Q-ZbRnAj8hkBXyPhT6jyZ-2vz Live Data: Name: shilpa9_99@_4KVK34WXWP2G9_G66zuktspUh1ha8eaFubLQB_2017-02-18 23:31:23txt Download Link: [login to view URL]!At8Q-ZbRnAj8hj8TStM0PrDgaajR Sql Query Code Download Link: [login to view URL]!At8Q-ZbRnAj8hj32ErBpxz31EIHI Part Two (Xero) Files: Data File Name Historical_Data_Demo_-_Company_-_(Global)_2017-02-18 23-33-18txt (1) Download Link: [login to view URL]!At8Q-ZbRnAj8hkEAaL0YhtMfnl4C Part Three Files (Deputy): Data File Name: Historical_Data_Test_2017-02-18 23-35-19txt Download Link: [login to view URL]!At8Q-ZbRnAj8hj7JBMpmJLe35EgS
ID del proyecto: 13226346

Información sobre el proyecto

9 propuestas
Proyecto remoto
Activo hace 7 años

¿Buscas ganar dinero?

Beneficios de presentar ofertas en Freelancer

Fija tu plazo y presupuesto
Cobra por tu trabajo
Describe tu propuesta
Es gratis registrarse y presentar ofertas en los trabajos
9 freelancers están ofertando un promedio de $161 AUD por este trabajo
Avatar del usuario
Hi, Answer: It doesn't matter if files are in same container or is separate container as long as file name can be identified using run time values (e.g. date, time etc.). I think the challenge with your project is to deal with dynamic number of "JournalLines" in historical in Xero. I have 4+ years of experience on Azure (total 11+) and I have been recently started working on DF. Please ping me to discuss more on your project. Thanks Prasoon
$250 AUD en 6 días
5,0 (3 comentarios)
4,0
4,0
Avatar del usuario
Hello, Im interested to work on this. i have gone through the high level job description. If you are interested, please let me know. We will discuss and start the project. Thank you. Regards, Sheik
$222 AUD en 7 días
4,9 (13 comentarios)
3,7
3,7
Avatar del usuario
Hi, I will work as per your direction. I ensure you the accuracy of my work. I am ready to start working right now.I will show you sample to make you sure about the quality of my work. I am waiting for your response to start working. Thanks
$30 AUD en 3 días
0,0 (0 comentarios)
0,0
0,0

Sobre este cliente

Bandera de AUSTRALIA
Canberra, Australia
0,0
0
Forma de pago verificada
Miembro desde ene 5, 2017

Verificación del cliente

¡Gracias! Te hemos enviado un enlace para reclamar tu crédito gratuito.
Algo salió mal al enviar tu correo electrónico. Por favor, intenta de nuevo.
Usuarios registrados Total de empleos publicados
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Cargando visualización previa
Permiso concedido para Geolocalización.
Tu sesión de acceso ha expirado y has sido desconectado. Por favor, inica sesión nuevamente.