Reinforcement Learning - Blackjack

Sutton and Barto describe a simplified version of the game of blackjack in their book "Reinforcement Learning: An Introduction". The learning process for Monte Carlo control with exploring starts is visualized using heatmaps of the state space.

View

Multi-Armed Bandits

The Multi-Armed Bandit explanation by way of a row of slot machines.

View

Mandelbrot Explorer

The Mandelbrot Set is a fractal constructed by iterating a formula and drawing the ranking for each pixel.

View

Chaos Game

The Chaos Game involves a series of attractors and the random movement of a point between them. This plotter lets you visualize ranges in magnitude when moving to the attractors.

View

Magnetic Sculpture

Intricate designs formed by combining many small magnetic spheres. Unlike beadwork where beads can be arranged in arbitrary patterns, the magnetic fields of the magnets interact in interesting ways and define a subset of shapes that are possible.

View

Fractalography

Images are generated using a custom written Mandelbrot fractal explorer. Designed to be used as backgrounds for dual monitor displays.

View

Morpion Solitaire

Morpion Solitaire is a pen and paper game where you try to get the highest possible score by completing lines of five connected points.

I spent extensive time on this problem and achieved the second highest world record score.

Dimitri Tishchenko

Photo of Dimitri Tishchenko

Software Engineer working for Waterworth