@good_faces_bot is probably the best Twitter account there is. It's pretty self-explanatory: just a lot of good faces posted regularly. I wanted to sift through the endless stream to find the best faces and this was the result: bryce.io/best-faces.

Get the faces 🐦

This part doesn't need much explaining, everyone with basic coding experience knows how to and probably has fetched from Twitter's API. Borrowing from the timelines quick-start guide I got an API token and downloaded all the tweets for the account.

After that, it sorts them and filters for tweets above a certain threshold of likes (1000). Then the last thing to do is to download the image associated with each tweet with a slight staggering to avoid getting rate-limited and it's ready to be analyzed!

Enter: rustface 🔬

Put simply, this is a Rust-y wrapper around the SeetaFace Engine, an excellent open-source tool for real-time facial recognition. It works surprisingly well given applications like this generally need massive amounts of processing power. Can read more about it here.

The included example was very helpful; I modified it to loop over the list of images and made it so instead of drawing a rectangle over the face it outputs a cropped image of the face. With some minor error-handling it was making it through all of the images in no time!

Unfortunately it only recognizes a face in about 1/3 of all the images currently (and some of the "faces" aren't actually faces, like the first one on the page 😂). With some tweaking this could be improved though.

Putting it all together 🪢

Now that the Rust part is complete, all that's left is to display the faces somehow. I used Svelte to build a simple site since I like using it for small projects like this.

The trickiest part here was that although the Rust step generated a bunch of images, there isn't an easy way to connect them back to each tweet. I used fs.exists to loop over the output directory and generate a bunch of imports & exports. It's hacky but ended up working well! But beware 👻 you shouldn't be generating raw javascript like this unless there are no other options, here's a snippet:

const hasOne = await fsExists(generatePath(tweet));

if (hasOne) {
    facePaths.push(generatePath(tweet));
}

facePaths.forEach(facePath => {
  output += `import ${faceFileName} from "../${facePath}";\n`;
});

output += `export const facesFor${tweet.id} = [${faceFileNames.join(',')}];\n`;

await fs.appendFile(outputFile, output);

Anyways once that was done I could loop over the list of tweets from the first step and display them in Svelte, referencing the facesForX import directly:

{#each tweets as tweet}
    {#if faces[`facesFor${tweet.id}`].length > 0}
        {#each faces[`facesFor${tweet.id}`] as face, i}
            <Face face={face} />
        {/each}
    {/if}
{/each}

Note that it checks if length > 0: since it isn't able to detect a face in every image, unfortunately some get left behind. It's sad too since a lot of the really good faces are the stranger looking ones.

Then I added a simple hover state to show the original image that the face was detected in:

<script>
    export let tweet;
    export let face;

    export let show = () => showOriginal = true;
    export let hide = () => showOriginal = false;

    let showOriginal = false;
</script>

<a href={tweet.tweet.text}>
    <img src={showOriginal ? tweet.media.url : face} on:mouseenter={show} on:mouseleave={hide} />
</a>

That's pulled directly from the Svelte component too! No wrappers or boilerplate, part of the reason why it's so satisfying to use for development.

Et voilà! The last kicker is that since all of these are just scripts, they can run sequentially in a GitHub Action! Although the facial recognition part takes a while (~20m) the simpler ones can fetch/generate the necessary data and run entirely in CI. There are some optimizations to be done here around uploading/downloading artifacts but in the meantime I'm happy it all works remotely.

Check out the code here:

npm install
npm run fetch:tweets
npm run sort:tweets
npm run fetch:images

cargo install
cargo run

npm run generate:imports
npm run dev