Full Machine Learning Project — Processing Raw Data (Part 2)

Want to get started with freelancing? Let me help: https://www.datalumina.com/data-freelancer Need help with a project? Work with me: https://www.datalumina.com/solutions In this video, we are going to read all the separate raw CSV files, process them, and merge them into a single DataFrame. 👉🏻 Source material for this week: https://docs.datalumina.io/s1eFpcdtbvRTMb ⏱️ Timestamps 00:00 Introduction 00:58 What we will cover in this video 02:00 Downloading the dataset 03:05 Open Visual Studio Code workspace 04:00 Creating the conda environment 08:05 Downloading the Python script 09:10 Understanding the data 14:46 Understanding the data transformation 19:12 Reading a single CSV file 23:29 List all files in directory 26:10 Extract features from file name 35:40 Read all CSV files 47:01 Working with datetimes 55:58 Turn code blocks into function 59:55 Merging DataFrames 01:06:00 Resample data (frequency conversion) 01:18:38 Final check 01:20:06 Exporting the dataset 01:22:04 Conclusion and summary Project overview (what you will learn) Part 1 — Introduction, goal, quantified self, MetaMotion sensor, dataset Part 2 — Converting raw data, reading CSV files, splitting data, cleaning Part 3 — Visualizing data, plotting time series data Part 4 — Outlier detection, Chauvenet’s criterion, local outlier factor Part 5 — Feature engineering, frequency, low pass filter, PCA, clustering Part 6 — Predictive modelling, Naive Bayes, SVMs, random forest, neural network Part 7 — Counting repetitions, creating a custom algorithm Link to playlist: https://youtube.com/playlist?list=PL-Y17yukoyy0sT2hoSQxn1TdV0J7-MX4K If you find these videos helpful, consider subscribing @daveebbelaar