Well Log Django App

I've been working on a website for working with well logs on and off for a while now.

Meet Loggie, a well log website!

With Loggie, you can join and create groups to share your logs, and customize how public or private each group is. It's similar to Reddit's permission system with well logs instead of posts.

Well logs are a collection of sensors lowered into a well bore. The variety of sensors, scale of the data, and visualization options create an interesting challenge for displaying the data.

At the moment the visualizations are based on a mapping I created of common logging tools and how they're plotted. An enhancement to that would be to allow users to customize the visualization, plotting tools Loggie doesn't recognize.

All the code I've written for Loggie is closed source at the moment, but I'm thinking I'll open source the Permission system app at least.

Here's a screenshot from an array induction log:

Array induction log

Here's the link to that log.

SQL Database of Directory Data

I was running out of disk space on my 128GB Surface Pro, so I needed to find out where all my space went.
I made a python package, directory_to_sql, to walk through a directory tree, gather file system data, and store it in a SQLite database.

The data retrieved includes:

• File sizes and modification date.
• Folder sizes and file counts.
• Recursive (including sub-folders) folder sizes and file counts.

The package contains several queries to find top 10 largest files and folders. This was all I really needed to solve my space problem. 

The package returns a database connection to the SQL database, so it's very extendable for further analysis. The package is published on PiPy and has Travis CI running tests on every push.

CS50's Web Programming with Python and JavaScript

I recently audited Harvard's online class, CS50's Web Programming with Python and JavaScript.
It was a 12-lecture course covering:

• Git 
• HTML, CSS 
• Flask 
• SQL 
• ORMs, APIs 
• JavaScript 
• Front Ends 
• Django 
• Testing, 
• CI/CD 
• GitHub, Travis CI 
• Scalability 
• Security

I posted three new repositories on my GitHub from assignments from the class. 

Book review Flask App

GitHub

Website where users can log in, search for a book, go to a book's page, see rating data from GoodReads, and add a review. 

Web Socket Chat App

GitHub

This is another Flask website. It's a chat room website. It uses flask_socketio, which is an Ajax library for flask. Users can send and receive messages and change chat rooms, all while staying on the same web page. 

Pizza Order App

GitHub
Users create an account and log in. The order page allows the user to add pizzas to an order, customizing the toppings, crust, and size. Staff can add their pizzas and track pending, unfulfilled orders via Django's Administration page.

Reading digital well log files with lasio

A well log is a measurement with respect to depth or time of various physical characteristics in or around a well bore. Well logs are used extensively by the oil and gas industry. The universal digital format of well logs is the Log ASCII Standard (LAS).

Reading and interpreting well logs has been a common task for me both in college and at work. I have a project in mind for an app to store, view, interpret, and share well logs in a web app, so I want to be more familiar with the file format.

This Notebook is my first attempt at working with well logs using Python. I'm using lasio as my LAS parser.

My goals with the project were to

• Test if lasio works consistently with hundreds of well log files.
• Analyze the usage frequency of well logging tools and how standardized the syntax is between well logging tools in terms of how the tools are named and described.
• Plot some of the gamma ray, resistivity logs.

No test of a LAS parser would be complete without plotting the log data. Images of plotted well logs are quite large. It's common to have more than one data point per foot depth for each tool and a well can be many thousands of feet deep. Below is a cropped example of a plotted well log. Full logs can be found in the git repo. 

Cropped plot of resistivity and gamma ray logs

Forum Website

I made my first web application! 

It's a forum website. Think, reddit, or a site where people post and comment on things. I chose a forum, because some of my ideas down the road involve a website where users collaborate.

I built the project using Django, a Python web framework. Django allows for fast development and is built using a "don't repeat yourself" philosophy which minimizes redundant code.

After going through the official Django tutorial and poking around, I wanted to make something by myself.

Let's see it

Here's the Github repository. To run it locally from Command Prompt or Anaconda, navigate to the manage.py file and enter "python manage.py runserver". 

If you don't want to do that, I do have a few screenshots.

Index Page

The index page will look familiar to a reddit user. The eye grabbing content is the title of each post. In smaller text is when the post was submitted, who posted it, and where they posted it.
In the dark blue header, is a quick link to some of the website's forums. A list of all the website's forums is also available.

Forum page

Each Forum's page lists the forum's topics sorted to show the most recent. The forum's description is in the sidebar. The URL to get to a forum's page is really easy to read. It's just /f/[forum_name].

Topic page

Each topic's page contains the topic's body, followed by posts from other users. The topic page inherits the sidebar from its forum.

Final Thoughts

I used Visual Studio 2017 (VS) as my IDE. I never used VS for development before, but I found it very useful for web development. I'll definitely use it again for my next Django project.

I was happy that the front end web development class I took didn't go to waste. Bootstrap really adds a lot. 

Cross Over Stock Trading Algorithm

Sometimes I wonder if or when I should buy a stock, and I'll get a stock trading algorithm in my head. Would it beat holding the stock? How risky is it?

One way to access that is testing the algorithm on past market data. Here, I do just that.

Stock Market Analysis

• Use API to pull stock data
• Methods to analyze stock data
• Crossover trading concept 
• Backtest crossover trading strategy

Crossover algorithm trading periods

While this Notebook follows another blog post, I made significant changes to the code and added better visualizations.

Stock Trading Algorithm Testing

• Test the Crossover trading algorithm on all the stocks in the S&P 500
• Analyze results 

Return ratio of holding stock vs algorithm for S&P 500 stocks 

Crossover algorithm often exceeding hold when hold results in loss

GitHub for this project

Disclaimer: This is in no way financial advice. I am not a financial adviser.  This post and code were done as a data analysis exercise only.

Google's Deep Dream

Google's Deep Dream enhances patterns in an image similar to the way your mind perceives a face in the moon, an algorithmic pareidolia. I recently found a Jupyter Notebook, published by Google, showing how to produce the Deep Dream visuals. I thought I'd give it a shot.

If you have never heard of Deep Dream, this GIF shows its effect on an image.

This GIF shows how the number of Deep Dream iterations affects the image.

GitHub is not able to show two of the GIFs I created in my notebook due to file size limitations. I included them in this blog post (one I converted to video).

This video was made by applying the Deep Dream filter, zooming in, and looping that 200 times. There are some noticeable differences between this video and the original GIF. 

Here's how I used the Deep Dream program. 

I'm happy with how it turned out. I was able to see the capabilities of Deep Dream by iterating through various settings. I was able to use Python to turn the generated images into labeled GIFs and tiled images. I think it turned out to be an effective way to display the data.

Installation

This Notebook uses Python Imaging Library (PIL) which has yet to release a version for Python 3.X, so I'll need to run the notebook on Python 2.7. Good thing Anaconda makes managing different Python environments easy. I wonder why Google didn't use Pillow instead and run the project on Python 3.

Installing Caffe was more difficult than I expected. Some issues I came across are:

• Instead of using a package manager, Caffe needs to be built with Visual Studio.
• There are multiple versions of Caffe for Windows available on GitHub. The two I tried were from Berkeley Vision and Learning Center (BVLC) and Microsoft. 
• When I tried to build Microsoft's Caffe, the build failed. This was due to the build requiring a debug version of Python. Oddly the installation instructions made no mention of this requirement. It's only mentioned in the .PROPS file where the build's variables are set. 
• I was able to build the BVLC version of Caffe, even though it mentioned the same debug requirement in the .PROPS file. 

The Microsoft Surface I'm running this on has an integrated GPU, which does not support NVIDIA's CUDA. I had to use a CpuBuildOnly build. At default settings it took about 5 minutes to render an image using most filters. I would look into a GPU build for a larger project.

GitHub link for this project

For more Deep Dreams, see this Instagram.