This is the first part of the video series where I will build a Data Analytics project using SQL, Python, and Google Sheets. In this Data Analyst Portfolio project, we will scrape the Otodom website data using the Bright Data tool and then load the data into the Snowflake database. In the Snowflake database, we shall analyze, clean, and transform data and finally build reports to answer some of the most common questions related to the property market in Poland. , we shall use Python and Google Sheets. Get data from Bright Data using the below link: https://brdta.com/techtfq This data analysis project will be built as a 3 video series, this is the first part of the video series Each video part will be released within 2 days of each other. Part 1 – Scrape data, Load data and Flatten data using SQL. Part 2 – Transform data using SQL, Python, and Google Sheets. Part 3 – Analyze, Clean and build reports using SQL. I am building this project using property listing data taken from the https://www.otodom.pl/ website using the Bright Data platform. However, a similar project can be built using any other dataset of your choice. Timestamp : 0:00 Intro 0:30 Problem Statements we are trying to solve in this project 1:00 Overall flow of project 2:41 Overview of tasks in different parts of this video series 3:38 Understand the Otodom website 4:36 Overview of Bright Data platform to scrape data 7:57 Exporting dataset to Snowflake 8:32 Create and set up a new Snowflake account 12:32 Install SnowSQL 14:31 Creating new database, warehouse, tables, stage, and file format 24:02 Copy data from stage to snowflake table 25:28 Understanding JSON data in Variant column 26:05 Flattening JSON to individual table columns 31:39 Deleting unwanted test data 32:41 Conclusion Download the scripts and sample dataset used in this video from my blog. below: https://techtfq.com/blog/sql-data-analytics-project-data-analyst-portfolio-project-using-sql-python-google-sheets Please do like and subscribe to the channel for more SQL, Python Also would love to read your feedback in the comments below. Disclaimer: This video series is sponsored by Bright Data. 🔴 My Recommended courses 👇 ✅Complete Data Analytics Bootcamp: https://codebasics.io/bootcamps/data -analytics-bootcamp-with-practical-job-assistance?refId=ujxkeg5e-n948-4cpg-fx9g-q0fvzczdyp ✅ Learn Power BI: https://codebasics.io/courses/power-bi-data-analysis-with-end -to-end-project?refId=4b1e1886-3252-4bf1-86f4-7d603dab692f ✅ Learn complete SQL: https://learnsql.com/?ref=thoufiqmohammed ✅ Practice SQL Queries: https://www.stratascratch.com/ 🔴 WATCH MORE VIDEOS HERE 👇 ✅ SQL Tutorial – Basic concepts: https://www.youtube.com/playlist?list=PLavw5C92dz9HQQ_COgGb7kf_1H8UWUBxO ✅ SQL Tutorial – Intermediate concepts: https://www.youtube.com/playlist?list=PLavw5C92dz9FD9XspliRM_HZM_jK7tkii ✅ SQL Tutorial – Advanced concepts: https: //www.youtube.com/playlist?list=PLavw5C92dz9GbmgiW4TWVnxhjMFOIf0Q7 ✅ Practice Solving Basic SQL Queries: https://www.youtube.com/playlist?list=PLavw5C92dz9F66P3GFo5O6nJ4DfZNhVsp ✅ Practice Solving Intermediate SQL Queries: https://www.youtube. com/playlist?list=PLavw5C92dz9EIYmNXJ8ZtQ1bmLIpt0SpV ✅ Practice Solving Complex SQL Queries: https://www.youtube.com/playlist?list=PLavw5C92dz9Fahr7taauUx5RnTfuGyL– ✅ Data Analytics Career guidance: https://www.youtube.com/playlist?list =PLavw5C92dz9FDsr995DjwCy9XNaIukb8P ✅ SQL Course, SQL Training Platform Recommendations: https://www.youtube.com/playlist?list=PLavw5C92dz9GqF1Q_zn8KnYvFgtEOh-2n ✅ Python Tutorial: https://www.youtube.com/playlist?list=PLavw5C92dz9FePZXIo4jpsupqp3 X0w2SW ✅ Git and GitHub Tutorial: https://www.youtube.com/playlist?list=PLavw5C92dz9EI1MPlqqDpe3bgr8RJZE0b ✅ Data Analytics Projects: https://www.youtube.com/playlist?list=PLavw5C92dz9HwT_uStsDdJbuBOt_XD0ch THANK YOU!
26 Comments
can anyone share the dataset here? i cant download it because i dont have a business email. i am trying using my personal email. its not accepting
Instead of replace you can use
json_data:price::VARCHAR(1000) as price
This will eliminate the double qoutes and convert data to varchar datatype.
Hii I have registered to brightdata. I am unable to export it to Snowflake. It is asking for payment details. Even for sample 1000 records do we need to pay ???? Orelse the other option is can we import csv or json data to snowflake from local by downloading it from brightdata. I see thats the only option right now
Wow that is worth watching and learning from learner. Thank you
This is so amazing…
HI Sir your all videos are just amazing.. while watching this video i have a query why did u download snowsql and performed the creation of database , warehouse and stage in snowsql.. this can be directly done in snowflake worksheet ???
Wow , I am from Poland, near Warsaw
I am not able to export the datasets as the datasets is not appearing in my dataset window even after downloading it
In regex replace / is not included … @30:08
Can you suggest a learning track for data engineering
No option to download full data.. only sample csv or json
Hello Thoufiq, thanks for this informative content!! I keep getting an error whenever I try to run the PUT command as instructed in the 'Load sample dataset file'. The error says 'file doesn't exist'. Please how do I resolve this? Will be looking out for your response. Thanks!
how to download the dataset? can someone please help me? Thank you.
But using free account, we are not able to export the data, not getting that option
Anyone done that part after downloading data I am getting payment option 🥲
Hello from a Polish subscriber! Hope you enjoy Warsaw. Thank you techTFQ for your continuous efforts and being a big inspiration. God bless you
Thanks a lot bro@techTFQ
You're amazing. Thanks for sharing the knowledge bro.
Please make a snowflake series …. Much needed….
Hey…! if someone have downloaded the whole dataSet, then provide us, it will be helpful.
Thank you Taufiq!! You have no idea how wonderfully you executed on this tutorial!! I sincerely appreciate what you have done here!!!
hi tfq, i am not able to see the downloaded file in bright data to send it to snowflake and i am stuck completely, please help me
8 minutes in and loving it already! Wanted to see some end-to-end projects before diving into SQL so my ADHD brain can have an idea what the big picture looks like. Thank you for the video! 🎉
I did not understand how to get the data from bright data. They are asking to purchase it ..should I purchase it ?
Thanks man for giving us these kind of quality content 👍👍
Hello Taufiq, I trust you're doing well. Could you please inform me if you offer any comprehensive courses related to data analysis, whether they are paid or free? I'm quite eager to enroll in such a course if available. Thank you.