• Hidden Markov Models

    Posted on Saturday, February 13, 2021

    A brief summary of hidden Markov models, including an explanation of methods for inference and estimation.

  • Receiver Operating Characteristic (ROC) Curves

    Posted on Friday, May 8, 2020

    Let's all take a moment to think about how to quantify the quality of classification models. There is a lot of confusion because of the way the topic is traditionally discussed. I want to be clear.

  • The Linked List Cycle Problem

    Posted on Tuesday, September 17, 2019

    I've been studying basic computer science concepts recently because I don't have a traditional programming background, but people like to ask these types of questions at interviews for some reason (I don't exactly understand why). In any case, some of the practice problems are kind of fun, like this one concerning linked lists.

  • Policies and Optimality

    Posted on Monday, July 15, 2019

    In this mini-lecture I continue looking at some of the basic concepts underlying deep reinforcement learning and control. I'll define what is meant by a "policy" and how we might characterize the value of a policy. Finally, I'll discuss Bellman's optimality condition, which lies at the heart of the dynamic optimization problems we hope to solve in deep reinforcement learning.

  • Introduction to Deep Reinforcement Learning

    Posted on Monday, July 8, 2019

    This is the first in a series of notes on deep reinforcement learning. I introduce the basic setting of reinforcement learning, describe environments, agents, and Markov Decision Processes, and provide some simple examples from OpenAI's gym package.

  • Welcome!

    Posted on Monday, July 1, 2019

    Welcome to my blog! I plan to make this the home for my notes of research and development, especially on the topics of deep learning and deep reinforcement learning. Look for lectures, reviews, tutorials, and other sundry commentary in the days to come.