Accessing Smartphone And Laptop Cameras In The Web With Javascript
Hello! Today I’m going to show you how to access the built-in cameras in-browser, on computers and mobile devices using Javascript.
In this series, I will show you how to set up a simple HTML page that accesses the front-facing camera, runs MediaPipe Facemesh to detect facial keypoints, and then shows a livestream using the detection features.
This post in particular will show you how to:
- Properly access a device’s camera through the Javascript
MediaDevicesinterface (phone or laptop) - Stream it live to an HTML Canvas for viewing
The Rundown
Demo
Find the demo here: https://facemeshdemo1.netlify.app/
Code
HTML: https://gist.github.com/kongmunist/83d4862406b3363dcc07199780d81bad#file-index-html
JS: https://gist.github.com/kongmunist/83d4862406b3363dcc07199780d81bad#file-facemeshsetup-js
Deep Dive
Motivation
With the advent of Tensorflow.js, machine learning has begun to move into the browser. No longer will you need to download 6 different Python packages that are each 500 MB to train machine learning models — now all the capacities of Python Tensorflow have been moved into a single <script> import on a webpage with Tensorflow Javascript, and take up only 1.1 Mb when imported.
With mobile devices becoming more powerful every year, it will soon be feasible for entire computer vision training and testing pipelines to exist in a browser. ML will become more accessible to anyone with a smartphone or computer. I want to make it easier for people to develop computer vision applications on mobile devices with this tutorial. Let’s begin!
Initial Setup
We’re going to start with a blank HTML page that just has a video (for the camera stream) and two <script> tags; one for the MediaPipe Face Mesh model, then another for Tensorflow Javascript (TF.js) itself. Recently, Tensorflow has been updating so fast that the main fork didn’t work with Face Mesh. They’ve since fixed it, but if it ever stops working just start…
