Python

Disaster Tweets Natural Language Processing

Intro I have a dataset of tweets, which includes whether they are referring to a disaster or not. The goal is to build a model that takes a tweet and predicts if it is a disaster. This could be useful during an actual disaster to ensure only the most relevant ones are shown to emergency responders. The full code for this project can be found on my GitHub: https://github.com/jamesdeluk/data-science/tree/main/Projects/nlp-with-disaster-tweets Exploring and cleaning the data I started by looking at the raw data in a text editor; as it was only a few hundred kilobytes, it was easy enough to do:

Wednesday, December 11, 2024 | 26 minutes Read

Customer Analysis Part IV: Brand Analytics and Elasticity

This is part four of a multi-part series. Part one, segmentation and clustering, can be found here. Part two, classification, is here. Part 3, purchase analytics, here. As before, the code below is simply snippets. The full code for this section can be found in the repo: https://github.com/jamesdeluk/data-science/blob/main/Projects/customer-analysis/ca4-brands.ipynb Intro At last we reach out final part in this series. Here we’re looking at brands, quantities, and elasticities. Data The first thing is to reimport the data, the same as part three. I also created NumPy array of prices I’ll use later for testing and simulation. We know the prices in our dataset range from 1.10 to 2.80; for nice numbers, I picked 1.00 to 3.00, in 0.01 intervals:

Sunday, December 1, 2024 | 20 minutes Read

Customer Analysis Part III: Purchase Analytics

This is part three of a multi-part series. Part one, segmentation and clustering, can be found here. Part two, classification, is here. This post contains incomplete code snippets. The full code for this section can be found in the repo: https://github.com/jamesdeluk/data-science/blob/main/Projects/customer-analysis/ca3-purchases.ipynb UPDATE 2024-11-26: During part four I noticed I had made a mistake in the Product Analysis section - I’d accidentally used a subset of the dataset when doing an analysis. Given part four is exclusively about product and brand analysis, I have moved the now-corrected section to that post.

Friday, November 22, 2024 | 20 minutes Read

Customer Analysis Part II: Classification

This is part two of a multi-part series. Part one, segmentation and clustering, can be found here. Code for this section can be found in the repo: https://github.com/jamesdeluk/data-science/blob/main/Projects/customer-analysis/ca2-classification.ipynb Intro Great, we have our customers clustered! But, hopefully, over time, we’ll gain more customers, and they’ll need to be assigned to an existing cluster. This is called classification. There are a few techniques for doing this. First, let’s remind ourselves what our current clusters look like by grouping the data and finding the means, as we did in part one:

Monday, November 18, 2024 | 19 minutes Read

Customer Analysis Part I: Segmentation and Clustering

This is part one of a multi-part series. Part two can be found here. Intro Customer analysis is one of the most important uses of data science. The better an organisation knows their customers, the better they can ensure their needs are met. This could be a supermarket providing special offers on a customer’s common purchases, a charity tweaking their messaging based on their’ donator’s personal interests, or a distributor using purchasing trends to determine where to open a new warehouse.

Wednesday, November 13, 2024 | 18 minutes Read

Predicting Belgium’s future solar PV generation: Part II

This is part two of a two-part series. Part one covered collecting and processing the data, then assessing the forecasts in the dataset. It also contains the link to the dataset and repo. This part covers my predictions, using machine learning. Intro OK, we’ve looked at the data, and we’ve seen how accurate their predictions are/were. Can I do any better? I’m taking three approaches: SARIMA, TensorFlow, and Prophet. A quick aside: Why do we want to predict the future energy generation? There are a number of reasons.

Friday, November 8, 2024 | 15 minutes Read

Predicting Belgium’s future solar PV generation: Part I

This is part one of a two-part series. This part covers collecting and processing the data, then assessing the forecasts in the dataset. Part two covers my predictions. Intro I have a friend who owns a solar management platform, and he asked me to investigate some solar PV generation data provided by Elia, one of Belgium’s transmission system operators. The data includes measured solar PV generation by region, along with their various forecasts.

Sunday, November 3, 2024 | 15 minutes Read

How much for my car? Using machine learning to find out

Intro I own a wonderful Ford Fiesta ST-3 (2017, 7th generation, 38k miles), but since moving to London, she’s been sat at my parents’ house. I finally accepted it’s best to sell her. But how much should I ask? Let’s do some data analysis and machine learning! The JupyterLab notebook, extractor script, and data, is available on Github: https://github.com/jamesdeluk/data-science/tree/main/Projects/car-price-regression-modelling Gather the data Auto Trader makes it difficult to scrape data - the page is loaded dynamically, and has robots.txt etc to restrict bots, so I couldn’t find an online tool that worked. They do provide an API, but that’s only for industry, and likely costs a pretty penny.

Wednesday, October 16, 2024 | 21 minutes Read

Making a website with ChatGPT: DiscussAnyPage

Summary I went from an idea to a live website using ChatGPT in only a handful of prompts and a couple of hours, writing no code myself, and only copy and pasting commands. URL: https://discussanypage.onrender.com/ The idea On Saturday night, I had an idea - what if there was a way to leave a comment on any webpage? Sure, sites like Reddit and Disqus are similar, but I wanted to implemented in a different way. From any webpage, a bookmarklet or browser extension would automatically create a page on a website (I called it DiscussAnyPage, or DAP for short) where anyone can leave comments specifically for that webpage.

Tuesday, May 21, 2024 | 5 minutes Read

Mental Maths Practise Game

A Python script for improving mental maths

Wednesday, March 8, 2023 | 1 minute Read

Seek.com.au Job Scraper

Which terms are popular in job descriptions? Where are the jobs? Enter your search term and find out. Click Run ► Drag the separator between the code and the output to the top to maximise output

Python

Thursday, July 2, 2020 | 1 minute Read

Australian Specified Work Postcode Checker

In Australia on a working holiday visa? Want to know if the place offering you work is in a regional area? Put in the postcode and find out here. And yes, it’s ugly - it’s a 5 minute GUI built with Flask (my first Flask project!) View on PythonAnywhere (external link)

Python

Saturday, May 30, 2020 | 1 minute Read