Posts about python

Solving the Lowest Common Ancestor Problem in Python

Finding the Lowest Common Ancestor of a pair of nodes in a tree can be helpful in a variety of problems in areas such as information retrieval, where it is used with suffix trees for string matching. Read on for the basics of this in Python.

Read More
streaming data
Streaming data with Flask and Fetch + the Streams API

Streaming can be a great way to transfer and process large amounts of data. It can help save space and/or time, if the data uses a lot of memory, or if you want to start processing or visualizing the data as it comes in.

Read More
Point in Polygon search with MongoDB

In a recent project, we had a large number of points on a canvas, where a user could draw a region of interest to see only the points within that area. Here is a demo of how to do that using MongoDB with a geospatial 2D-index. Visualized using D3.

Read More
Image Similarity with Python Part II: Nearest Neighbor Search

This is Part II of my post on image similarity in Python with perceptual hashing. In this post, we will use Spotify's Annoy library to perform nearest neighbors search on a collection of images to find similar images to a query image.

Read More
cloud grid
Deploying a Flask app on AWS Lambda with Zappa

In this post I wanted to document the steps I took to deploy a Flask app to AWS Lambda using Zappa. Configuring everything properly in AWS was the most complicated part, so I hope this post can help other AWS noobs who might be struggling!

Read More
Accessing the Spotify API with Python

You can do a lot of interesting things with the Spotify API, like searching for artists and playlists, following and sharing them, and more. In this post we will access the API using Python to get featured playlists and associated artists and genres.

Read More
Playing Minesweeper graphic
Solving Minesweeper in Python as a Constraint Satisfaction Problem

Let's play Minesweeper in Python. In this post we will treat Minesweeper as a constraint satisfaction problem and use common algorithms like constraint propagation and backtracking search to mimic logic we would use to play the game as humans.

Read More
Generating Minesweeper boards in Python

In the next couple of posts we're playing Minesweeper in Python. You may be familiar with it since it probably can be found on your nearest computer. First we need to generate a board - that's this post - and then in the next, we will play the game.

Read More
Flood fill graphic
The Flood Fill Algorithm in Python

The flood fill algorithm has several high profile uses, most notably the bucket fill tool in image editing programs, as well as in games like Minesweeper. In this post we will go over how the tool works, as well as how to implement the algorithm.

Read More
Accessing the FRED API with Python

FRED is a database with time series data on economic indicators from a wide variety of sources. There is an API to access all of this data, and in this post I will go over a recent project where I needed to collect all of it.

Read More
Arc Diagram Graphic
Arc Diagrams in D3.js: Visualizing Taxi Pickup and Dropoff Data

An arc diagram is a type of network graph where the nodes lie along one axis, with arcs connecting them. This post is part one of two, where we will prepare the data to visualize pickup and dropoff locations for ride hailing app rides in NYC.

Read More
Time Series Data In Pandas: An Introduction

Time series data is all the rage these days, and not just in fields like finance. In this post we will look at working with time series data in Pandas, how to do basic time-based manipulations and calculations such as rolling means and data shifting.

Read More
using word vector features with scikit-learn (featuring spacy)
Building a custom Scikit-learn Transformer using GloVe vectors from Spacy as features

Word vectors are useful in NLP tasks to preserve the context or meaning of text data. In this post we will use Spacy to obtain word vectors, and transform the vectors into a feature matrix that can be used in a Scikit-learn pipeline.

Read More
Python Maze Generator Part II: Voronoi Diagrams

Voronoi diagrams are used in a variety of fields for a variety of reasons, including the art and design world. This post is Part II in a series on mazes, where I will generate and solve random mazes from Voronoi diagrams using Python and Matplotlib.

Read More
Rihanna coreference resolution paragraph graphic
Coreference resolution in Python with Spacy + NeuralCoref

Coreference resolution is a task in Natural Language Processing that aims to group together all references to an entity, for example, a person like Rihanna, in text. In this post we use NeuralCoref - a Spacy extension - to do this in Python.

Read More
Convex hulls in Python: the Graham scan algorithm

Computing the convex hull of a set of points is a fundamental problem in computational geometry, and the Graham scan is a common algorithm for it. In this post we will implement the algorithm in Python and look at interesting uses of convex hulls.

Read More
Determining how similar two images are with Python + Perceptual Hashing

Years ago I had an app idea where users could upload an image of a fashion item like shoes, and it would identify them. In this post I will go over how I approached the problem using perceptual hashing in Python with Pillow and the imagehash library.

Read More
Text Normalization for Natural Language Processing in Python

Text Normalization is an important part of preprocessing text for Natural Language Processing. There are several common techniques that we will go over in this post, using the Natural Language Toolkit (NLTK) in Python.

Read More
Python Project: Which cocktails can you make from a list of ingredients?

For many of us, going out to restaurants and bars is but a distant memory, and you might want to make your own cocktails at home. In this post we will build a program in Python to tell you what cocktails you can make from a list of input ingredients.

Read More
Accessing the Google Analytics Reporting API (V4) with Python

How to access the Google Analytics API with Python and create reports with your analytics data. This API seems complicated at first, but once you get the hang of how things work it's easy to generate new and interesting reports.

Read More
Generating and solving Sudoku puzzles with Python

You might be familiar with Sudoku - the single-player puzzle that involves inserting the numbers 1-9 into a grid in a certain way. In this post we will generate and solve Sudoku puzzles with Python using a depth-first search backtracking algorithm.

Read More
Feature Engineering with Python + Pandas: An Introduction

Feature Engineering is an important skill in data science, and is the process of taking raw data and turning it into features that can be used as inputs for training machine learning algorithms. We will look at 311 noise complaints data in this post.

Read More
Data cleaning with Python + Pandas: An Introduction

Cleaning up dirty, corrupted data with Python and Pandas. Dirty, corrupted data leads to dirty and corrupt analysis and conclusions. Who wants that? In this post we will go through a cleaning checklist with Pandas and a dataset from NYC Open Data.

Read More
How to train a custom Named Entity Recognizer with Spacy

In this post we will train a custom Named Entity Recognizer in Python with Spacy. I will go through the steps to prepare your data and train a model with it. Inspiration credit: text for the graphic is from Vogue magazine - link in post.

Read More
Point in Polygon search with GeoDjango

Determining if a point lies in a polygon is a pretty common task in computational geometry. In this post we will use it to answer questions like 'which NYC neighborhood is this apartment building in?' using GeoDjango and data from NYC Open Data.

Read More
Accessing NYC Open Data with Python + the Socrata Open Data API

If the walls in NYC could talk, they would likely tell you a similar story as one you can glean from 311 complaints. Noise complaints, building complaints, rat sightings, etc. NYC Open Data provides us this data, which we can access using Python.

Read More
NER graphic
Named Entity Recognition in Python with Stanford-NER and Spacy

Named Entity Recognition is a common task in Natural Language Processing that aims to label things like person or location names in text data. Today we will look at two examples in Python, using the popular libraries Stanford NLP and Spacy.

Read More
Anatomy Of A Web-Scraping Robot

What is a bot? Robots are bad, right? Not always. At its core a robot is just a program to automate various things you could do as a human, such as visiting websites. I will outline the parts, or anatomy, making up such a robot in Python.

Read More
Make a Twitter Bot with Python and Tweepy

How to access the Twitter API with Python using the Tweepy library. I will demonstrate how to connect to the API and do regular Twitter things like tweeting, following and favoriting, all using the API.

Read More
Classifying Fashion Articles with Python and Scikit-learn

Text classification is a popular and important problem that we deal with on a daily basis. I will be creating a text classifier with Python and scikit-learn to filter a collection of articles based on whether or not they are fashion-related or not.

Read More
What is web scraping?

Web scraping can mean a lot of things, but it usually refers to writing a program to visit websites and extract information from them. It can be a great tool when you need customized data, and I will demonstrate this with a scraper written in Python.

Read More