Real Time Jigsaw Puzzle Co-Op Website

While needing a quick break from developing, I like to solve a jigsaw puzzle or two, usually using the website Jigidi, and I felt intrigued by the idea of competing to see who could solve it the fastest. Jigidi already allows you to do so, but it’s hard to find puzzles a lot of people have solved/a lot of competition. I even tried contacting them about it, requesting a simple list of the most solved puzzle, but that was not in their interests for some reason. So I decided to give it a try myself.

I wanted a challenging solution to develop, so in addition to better search and leaderboards, I also wanted to add real-time co-op/collaboration so multiple people could solve the puzzle together.

But before I got to that, I started by making the core of the application, the jigsaw-solving window. I tried going with different 2D JS Game engines, but all of them seemed to be a pretty big overkill when looking at what I needed it for. So I simply used an HTML5 Canvas in 2D Context.

To generate the different shapes, I developed a generator that outputs a list of SVG-Paths, which then could be displayed on the canvas and the picture put on top using the masking feature of canvases. After polishing it for a while I was pretty pleased with the result as it felt responsive to the user, while still be configurable in many ways, such as snapping distance, multiple shapes, puzzle and so on.

After having dealt with SVGs, collision detection, etc. I needed a fun feature to work on, so I wanted to integrate one of AWS’s APIs into my application, mostly as I had a relevant class on it at the time. After looking around I choose to go with the Rekognition API, as it allowed to check pictures for any explicit content such as nudity and denying it if any was found with a certain confidence. In addition to checking for explicit content, it also returned a list of tags of objects present in the picture, for example, if it was a picture of a car, it would tag it as the car, maybe the settings it was in and so on., each of these tags are then saved and used for search, meaning the user could search for puzzle without the uploader having to tag it themselves.

Then I started working on the collaboration feature, and early on I discovered it would be a pretty big task since most of the jigsaw logic was written in the client and now had to be done server-side. So after converting it over to C#, I had a functional prototype, but most of the puzzle-generator didn’t get converted, so for now, all the bricks are square. The client receives all the shapes from the server, in unknown order and without knowing where the bricks should be placed, hopefully preventing a little bit of cheating.

But given almost everything the user did had to go through the server, a noticeable delay was inevitable and that’s not acceptable in a competitive game, so while researching ways to circumvent it without given the client too much information about the game, which would allow cheating, I got busy with some other things and the project stood still.

To conclude it, I developed a working prototype with the features I originally wanted which was collaboration, search, and leaderboards.

List of features:

Screenshots:

Frontpage of the website, where the user can discover recently uploaded puzzles.
The page the user sees when pressing a puzzle, including a preview generated by the server which updates every time the user changes the configuration.

 

The in-game view, where multiple players can collaborate in solving a puzzle. To the right a real-time chat can be seen, also giving the user updates about players leaving and entering.
The primitive ‘create a puzzle’ page, where the image is sent to AWS’s Rekognition API before being saved to moderated it for explicit content.
Prototype search page

Lifegr – Quantified Self Web-App

I always forget which places I have visited and traveled too, so I wanted to keep a log of those locations. Initially, I tried using a custom Map under Google Maps, but it’s limitations quickly made it hard to search and visualize.  So for fun, I created my own little personal-tracker I called Lifegr (Life-Manager).

I started out by using the Google POI API to gather information about places and Firebase for storing the locations I visited. But, as with my Home Controller Project, I prefer to be in control of my data and be able to host it locally if I want to. So I imported my data from Firebase and into a locally hosted database.

But as the project slowly grew as I wanted more features, one of which was 24/7 location tracking of my phone, I ended up renting a DigitalOcean Droplet and hosting it there.

Features

The app is still being developed when I’m missing a feature, but here’s a small curated list of features.

Phone Tracking

Whenever I sat down and started noting all the different places I had been, I sometimes found it hard to verify when and sometimes where I had been. So I wanted to track the location of my phone as it’s with me most of the time.

To solve this issue I looked for an app, preferably open-source where I could enter my own server IP-address and have it push the location and status of my phone to said server. Luckily I found OwnTracks, the exact app I was looking for, but the only ‘issue’ was that it pushed over MQTT, so I wrote a small app that subscribes to the MQTT topic and writes it to a separate database.

Additionally to GPS location, OwnTracks also pushes battery percentage, if my phone is connected to wifi, etc. But for now only GPS Location is visualized:

File upload

On my study abroad trip I wanted to keep track of my expenses and receipts. But, if I for some reason needed to retrieve a receipt at a later point, I wanted to keep an image copy of it.

So some kind of file hosting was needed, I solved it by using an S3 (Amazon Storage) Compatible self-hosted solution, where I found Minio to be a pretty good fit. The file gets uploaded through the application server, where meta-data about the original file such as filename, uploaded data, etc. and it is then available for viewing by the user:

For each uploaded document I could add meta-data about how much money was spent on the transaction, location and what the time was (Including Timezone).

Personal Info

Then I started researching my family tree a little bit and thought it would be interesting to create a family tree, so I did:

I can click on each member and see their auto-generated family tree as well.

Since I had all the location data on myself, I wanted to generate I list of countries, municipalities, etc. I have visited, so I found datasets online for the different areas, which available in GJON (Geo-Json) and implemented an algorithm I found online to iterate over all unique places I have visited and calculated if it was inside any borders, which I then displayed:

This is a feature I will properly come back to some point to include Phone GPS locations and display at what times the user visited those locations, which shouldn’t be too difficult given the data is all there.

Even without phone location, it does take a few seconds to run through all the places I’ve visited and see if it is inside one of the ~4000 different borders/areas I’m currently checking against. So some optimization might be needed at some point.

 

To be continued.

FoodSidekick

The following post contains snippets from my Bachelor Project’s Report. The project/website was called FoodSidekick and was made in collaboration with Cecilie Moriat.

Introduction

[…]

Up to 75% of the average diet’s climate impact derives from meat and dairy products whereas the largest single contribution is beef. This is mainly caused by the livestock emitting significant amounts of methane and the need for a large amount of fertilizer and area to produce the feed.
Thus altered consumption patterns in favor of commodities with a reduced climate impact are deemed necessary to significantly reduce the food sector’s GHG emissions.

[…]

On the basis of the above-mentioned reports, it is assumed that an uncharted potential exists in a climate-informative software platform which could help consumers to be aware of their everyday greenhouse gas emission caused by their diet.

Solution

In order to facilitate an informative experience for users regarding their climate impact, FoodSidekick was developed. This is a web application where users can register and manage an account and connect to friends, family and other users of the application.

[…]

A web feed is used to provide the user with frequently updated content from connections, which could make the user discover new recipes. Users can also upload their own recipes. FoodSidekick then provides an overview of the summarized amount of nutritional values and the environmental impact measured in CO2 equivalents for each recipe and ingredient.

The solution is built upon the three-tier system architecture, where the system is parted into three tiers.
The front-end consists of an Angular application written in Typescript, the back-end is an ASP .NET Core project which utilizes ElasticSearch. This project can be horizontally scaled as it is containerized. Data is persisted into MongoDB, a NoSQL document database.

Screenshots

feed.png
Feed

Dis

discover.png
Discover
recipe_env_impact.png
Estimated nutritional values

Smart Home Controller

Every day more and more devices are being added to the average home, these devices can help monitor and make life easier at home. They may monitor temperature, register activity, control lighting etc. But having multiple apps across multiple devices, some of which requires working internet, complicates the user and in turn the overall user experience.

A friend of mine, Steffen and myself started talking about a solution to all of this, a self-hosted hub/website that would connect all devices and allow for some degree of automation. At the time we looked around and couldn’t find a solution we liked or deemed mature enough, so we tried to make our own in the hopes we would learn something useful along the way.

Steffen focused on creating an event-based scripting language to be used for automation. And I started working on the server/hub, I choose C#/.NET for the language as it’s quick to develop in and relatively easy to maintain readable. For the UI, Angular 2 was used.

As the project pretty quickly gained basic functionality we needed some devices to control, but at the time none of the devices I had, had .NET SDKs (Sonos, KNX & Philips Hue). So I made open-source libraries for all of them, with most functionality used on an every-day basis. Out of all of them, KNX was properly the most challenging protocol to get working as it was a session-based bit/byte based UDP protocol, but after a while I got it working and a few people have even used it to my knowledge.

At this point, we had a working alpha/beta with the scripting language, basic house-setup, plugin-system and more, here’s a small demo of it I found in my archives, where it for the first time controls a Philips Hue Bulb:

Then .NET Core was announced and the first alpha/beta was released, and straight away I tried making the switch so I could start running the server on Linux as well. But given the early stages of .NET Core a lot of issues occurred, from loading DLLs at runtime was no longer possible (Which we used for our plugins) to the solution would not build consistently for no apparent reason. So the project stagnated for a while.

As the .NET Core issues slowly got solved with updates, the development continued for a while. But it became apparent that the solution would be too big to continue to develop for a two-person team, especially as device API’s changed often.

In the end, we managed to create a working solution, connecting multiple different devices, and looking back at it, our solution actually ended up pretty similar to another solution, which gained traction while we where developed our solution, OpenHAB, a Java-based home automation hub, with a similar web-interface and underlying data-structure, confirming to ourselves that some of our design choices weren’t too bad :-).

Conways game of life

Something most programmers should do is to implement a version of ‘Conways Game Of Life’. It a fun little simulation consisting of a grid, where each position can be dead or alive. And with just a few rules a small universe is created.

  1. Any live cell with fewer than two live neighbours dies, as if by underpopulation.
  2. Any live cell with two or three live neighbours lives on to the next generation.
  3. Any live cell with more than three live neighbours dies, as if by overpopulation.
  4. Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.

– https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life

Publish ASP. NET Core health checks to AppMetrics

AppMetrics is a library that allows you to track different metrics in a .Net application, it can be everything from how many users are signing up to active socket connections. The metrics can then either be pushed or pulled using different storage solutions such as Prometheus.

If you want to include health checks in your metrics and don’t use AppMetrics Health which has built in support for reporting to AppMetrics you can use Microsoft.AspNetCore.Diagnostics.HealthChecks and the reporting service I wrote, which is based of AppMetrics Health’s HealthReporterBackgroundService:

This is a snippet of my actual implementation as I’m using NodaTime and abstractions for the Timer to make it easier to test, but I hope it’ll serve as a good base for your own implementation.

I opened a discussion issue for it here https://github.com/AppMetrics/MicrosoftExtensions/issues/12 so at some point it will hopefully be a part of AppMetrics.

– Jonas