Want to build a website that uses face tracking? Well, you’re in the right place!

Photo by Lagos Techie on Unsplash

Today I’m going to show you how to set up a webpage that does real-time face tracking in any browser, mobile, or desktop. I’m doing everything in HTML and JavaScript to allow for cross-device portability. I’ll also show some easily extended working demos for face tracking, a basic Snapchat filter, and scientific applications.

All demos can be found here: facemeshmedium.netlify.app/

About the Model

With Google’s rapid development of Tensorflow.JS, machine learning on the web has never been easier. We’re going to be using Google MediaPipe’s Face Mesh model for all of our face-tracking needs.

Hi! Today we’re going to be comparing the usability of various household devices (Microwave, TV remote, keyboard, etc.) against those of research devices (like eye trackers and brain-computer interfaces).

Information Transfer Rate and Bits/Min

To do this, we’ll be investigating each device’s information transfer rate (ITR), or how fast the user can communicate with the device. We have some intuitive sense of this — obviously the keyboard allows for faster and more varied input than the TV remote. It’d be easy to compare the speeds if they had the same number of buttons, but since they don’t, we have to define some common metric to…

Hello! Today I’m going to show you how to access the built-in cameras in-browser, on computers and mobile devices using Javascript.

In this series, I will show you how to set up a simple HTML page that accesses the front-facing camera, runs MediaPipe Facemesh to detect facial keypoints, and then shows a livestream using the detection features.

This post in particular will show you how to:

  • Properly access a device’s camera through the Javascript MediaDevices interface (phone or laptop)
  • Stream it live to an HTML Canvas for viewing
Mobile camera livestreamed to an HTML Canvas

The Rundown

Find the demo here: https://facemeshdemo1.netlify.app/

HTML: https://gist.github.com/kongmunist/83d4862406b3363dcc07199780d81bad#file-index-html

JS: https://gist.github.com/kongmunist/83d4862406b3363dcc07199780d81bad#file-facemeshsetup-js

Deep Dive

With the advent…

A beautiful but expensive piece of electronic test equipment

Hi! Not everyone can afford the $2000 for a beautiful function generator like this Agilent 33210A, but everyone deserves to have basic function generation capabilities. So, today I’m going to show you how to make a basic function generator using everyone’s favorite microcontroller: an Arduino Uno!

In this post I’m going to go over some basic code, and how to set up an LED to test it out.

Hardware setup

Set up the LED+resistor with positive lead in pin 9 and the negative plugged into ground. Plug the Arduino into your computer.

TL;DR: Demo here. Hit the button at the top to start.

I recently tried out a bunch of different Javascript accelerometer demos that claimed to able to access the sensor in-browser. It was usually a simple demo: dots rolling across a screen due to gravity, or perspective shifting a 3D scene based on what your orientation was. However, none of these demos worked at all — no popup asking about permissions or anything. What gives?

In iOS 13, Apple added an option under Safari to allow webpages to request accelerometer access. However, the webpage couldn’t just spam you with a…

Hi, Andy here. I want to bring your attention to the depressing fact that there are really only two ways to sell software. Well, three, but hardly anybody uses the third. I am, of course, talking of subscription services (SaaS) and advertisement/data driven.

Software as a Business Model

Before people could write software for a living, you had to sell things. Tangible objects that performed a certain function, whose parts cost money, which further cost more money to assemble into a product.

Two people exchanging a bag of money for a house
Two people exchanging a bag of money for a house
Transactions before software

After computers became widespread people began writing software, and realized that other people would pay real money to buy your software. And best…

It happened again. You didn’t notice the gloomy skies as you pulled out of your driveway to go to Home Depot, and now you’re umbrella-less and sitting in their parking lot, watching the raindrops bead up and flow down your windshield. Worse yet, their New Grill November sale has summoned every dad within a 25 mile radius to the store, forcing you to park in the back of the lot. You didn’t even bring your rain jacket, because it’s new and you didn’t want to dirty it while carrying your new oak floorboards.

As you sit there feeling sorry for…

“Waking up one morning, a student of Zen realized he was going to be late to meet his teacher. He quickly got up and rushed out to his teacher’s hut. Upon arriving, he took off his shoes outside and greeted him.

Noticing his shortness of breath, his master asked ‘Which shoe did you put on first this morning?’ Ashamed that he did not know, the student realized he was not ready for his lessons and went home.”

Every time I’m late, I’ve always been working on something. I see the time, throw my shoes on and rush out. Along the…

Bear with me. I’m going to try to explain something slightly abstract but incredibly interesting.

Apparently you can’t embed a mirror in a blog post

You, the brain reading this

Look at your hands. Admire your reflection in a mirror. That’s you, right?

Wrong. That is not you. You are the brain inside that person. You are a chunk of meat piloting a bone-armored meatsuit, using your limbs as tools to let you interact with the physical world. And they’re our best tools because they’re so generalizable; I’m currently using the fingers on mine to hit my keyboard in specific combinations to write the words you’re reading.

However, their generality leads us to create more…

Hello! This is Andy. I’m about to tell you about something way cooler than implanting a chip in the brain to control robot arms, which is a very hard subject to top.

You can read brain activity by just shooting lasers into your skull and seeing how much of it comes back out. Introducing fNIRS, or functional near-infrared spectroscopy.

The basic principle of fNIRS is that blood changes shade as it becomes more oxygenated. Since deoxygenated blood absorbs more red light than oxygenated, it appears darker.

Not so blue, right?

You can think of fNIRS as shining a flashlight onto the brain. Since groups…

Andy Kong

Hi! I’m Andy. I try to make things that haven’t been made before. Check out my personal projects at https://andykong.org/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store