WWDC 2021 - Keynote

June 7, 2021

I was an apple fan for the last 5+ years and I’m always eager to watch their presentations. They always have marketing-driven demos, but in this year they outbullshitened themselves. Over the many California style stages and inspiring fake smiles it’s getting harder than ever to see actual features, that will help me personally in everyday life:

• Notification profiles and, especially, autoanswer in iMessage, that user is in focus mode.
• Tab groups in safari. Instead of several windows use group tabs - that can save screenspace on small laptop screen and also focus on one topic. • ‘async/await’ in Swift is long overdue. I didn’t wrote in Swift for the last 5 months, but I may come back one day and it would be nice to see these features, that were implemented in C# a decade ago.

That’s actually all news that I really care about. iPad mulitasking and Playgrounds enchantments looks promising, but I am not active iPad user. Not every wwdc should be amazing (like last year was with widgets, app library and  silicone macs), today is a quiet one. With is good, I just wish that presentation were more actually inspiring and less from pinkish kindergarten world.

JsonConverter and enum flags

June 6, 2021

It is beginning of June and it means, that time for first post of the year has become. Enum flags are useful wrapper around bitmasks and can be used to store several flags into one field/property. Internally they are just integers where each bit represent a flag state and can be set on or off. For example, lets define what next weekend should look like by writing down possible plans:

[Flags]
public enum WeekendState
{
    NONE = 0,
    SEX = 1 << 0,
    DRUGS = 1 << 1,
    ROCK = 1 << 2,
    ROLL = 1 << 3,
    DONUTS = 1 << 4
}

Here each state have their own 1 bit in place and can be combined with every other one. By adding Flags attribute we tell .net to treat this enum as a bitset.

And let’s make simple controller, that will predict our (hardcoded) future:

[ApiController]
[Route("[controller]")]
public class WeekendController : ControllerBase
{
	[HttpGet]
  public WeekendState Get() =>
	WeekendState.DRUGS | WeekendState.DONUTS;
}

If we call this controller we will get 18. Donuts is 10000 aka 16, Drugs is 10 aka 2. 2 + 16 = 18, easy. But if you don’t see backend code it can be not so easy. I’m asking second main question of the universe - what should I do on weekend and I’m getting 18? What doesn’t this mean? ¹

We need to serialise enum flags as strings. Easiest way to do that - add json serialised options in Startup.

services
    .AddControllers()
    .AddJsonOptions(options => 
        options
            .JsonSerializerOptions
            .Converters.Add(new JsonStringEnumConverter())
        );

Here JsonStringEnumConverter is default way of enum serialization. After these changes our little endpoint will return “DRUGS, DONUTS” which is an improvement. But frontend (or whoever will use api) may not like string with separated with comma values. The most logical way would be to return the array. To do that we need to write our own JsonConverter.

public class FlagJsonConverter<T> : JsonConverter<T> where T : struct, Enum
{
    public override T Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        //TODO
    }

    public override void Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options)
    {
        //TODO
    }
}

And use it by default for type WeekendState :

services
    .AddControllers()
    .AddJsonOptions(options => 
    {
        options
            .JsonSerializerOptions
            .Converters.Add(new FlagJsonConverter<WeekendState>());
        options
            .JsonSerializerOptions
            .Converters.Add(new JsonStringEnumConverter());
    });

Note: Order here is important, first passable converter will be used.

First method in our new converter is Read, which takes values from Utf8JsonReader and does everything possible to convert them into required enum (converter class is generic and can be used for all enums).

var derserialised = JsonSerializer.Deserialize<string[]>(ref reader);
var textValue = string.Join(',', derserialised ?? new string[0]);

return Enum.TryParse<T>(textValue, ignoreCase: true, out T result)
        ? result
        : (T)Enum.Parse(typeToConvert, "0");
}

Input string for that is parsed json token, which we convert to string array (if possible) and then join back as comma-separated strings. May be there are more elegant solution, but this was a fastest to code. And then this string is parsed using default parser. If failed - it tries to parse zero value, which lead to simplest result (WeekendState.None).

Another function is Write:

writer?.WriteStartArray();
var values = Enum.GetValues<T>().Where(e => value.HasFlag(e));
foreach (var val in values)
        writer.WriteStringValue(val.ToString());
writer?.WriteEndArray();
}

This is taking existing value, iterate it into all assigned flags and convert them into strings. And, on the way wrap it into array.

And right now result of our beautiful endpoint is:

[
  "DRUGS",
  "DONUTS"
]

That was exactly what I need.

And for final step here are xunit tests for deserialisation:

[Theory]
[InlineData("[\"DONUTS\"]", WeekendState.DONUTS)]
[InlineData("[\"ROCK\", \"ROLL\"]", WeekendState.ROCK|WeekendState.ROLL)]
[InlineData("[]", WeekendState.NONE)]
public void TestDeserialisation(string input, WeekendState result)
{
    var utf8JsonReader = new Utf8JsonReader(Encoding.UTF8.GetBytes(input), false, new JsonReaderState(new JsonReaderOptions()));
    var converter = new FlagJsonConverter<WeekendState>();
    var value = converter.Read(ref utf8JsonReader, typeof(WeekendState), new JsonSerializerOptions());
    Assert.Equal(value, result);
}

Bonus: spending each weekend eating donuts and drugs can be tedious. Let’s update controller code:

[HttpGet]
public WeekendState Get() 
{
    Random rnd = new Random();
    var shuffled = Enum.GetValues<WeekendState>().OrderBy(c => rnd.Next());
    var values = shuffled.Take(rnd.Next(shuffled.Count()));
    var result = WeekendState.NONE;
    foreach (var val in values)
        result |= val;
    return result;
}

It always contain random amount of random items. It also always include NONE value, which is not a proper solution.

HexThrees

December 11, 2020

Today I'm releasing my first app for iOS, that I have developed: HexThrees!. This is a small puzzle game, that was inspired greatly by 2048 and Threes!. I have played 2048 for a long time in the past and one day a question came into my mind: what would happen, if a game field would be a hexagon? Overall, hexagons are Bestagones! This ideas were collided with fibonacci numbers, that we have used in kanban planning weekly at work.

I wanted to create simple, easy mind game, that can keep my hands busy, when I listen to podcast or talk by phone. Merging rules might look overcomplicated at first glance, but after a while I found them much more interesting and fun to play with comparing to 2048 and Threes!

Although initial commit was made in april 2018, there were big gaps in development due to other projects, family, my real job and lazyness. Most of the game logic, graphic effects and base functionality was written in a train during commute between small town where I live and Hamburg where I work. But 2020 finally gave me opportunity to finish this game, polish it (as far as time allows me) and publish.

And this last mile was especially rough. I knew, that last percent takes more time than any other percent, but this feels especially frustrating because I kept promising to myself to get rid of this project as fast as I could and not waste more time on a hobby, that will not bring any amount of money. But still, the least stuff was left to do, the more it seems it took time. Iap integration, l10n (only english and russian), stupid bug in tutorial¹, screenshots for all sizes (hello, hardcoded save!), preview video (hello, ffmpeg!), first app store rejection and more small issues. Tasks, that I estimated for couple hours, required a day, and when I though, that I can deal with all in a couple days, it actually took a couple weeks.

But now it's over. All I left to do is to finish this post, push "publish" button, write some messages for reddit and relay.fm discord, where someone might be interested in that kind of game, publish instagram post and relax for a while. Hopefully, there would be no significant bugs².

This is a weird year. I (kinda) lost my main job, run 5k for the first time in summer, then 15k a month ago, which would be totally insane for me a year ago, and now this. I was writing small game prototypes since 2012³ and always wanted to release something for real. Today is the day.

Json to classes

October 22, 2020

Just a little thing, that I learned today. Here is the usual example: someone gave you example json for some API, that you need to implement and first step would be – created to classes for this json. There is a nice feature in Visual Studio: copy json in clipboard, then you can Edit -> Paste Special-> Paste JSON as classes.

One problem for me was that all fields were lowercase. Fixing them manually was too boring, so I wrote regex to replace first letter to uppercase: (\bint|\bbool|\bstring|\bfloat\bDateTime) ([a-z]) <- replace this
$1 \u$2 <- to this

No sure, that it will be useful for someone, I’m just proud of myself each time when I can use regex 🤓

If you are reading this, that mean that I have ..

September 1, 2020

..deployed this blog to azure web apps successfully. But it was a bumpy ride there, let me tell you.

how it was before

I have started this blog as a sandbox project and first things that I have learned in 2017 were: how to build docker image, push this image to repository and how to build dotnet core application. Therefore, every new version that I build, I started manually this scripts:

• build.sh

SERVICE="hysite"
pushd ./Hysite.Web
dotnet publish -c Release
popd
docker build -t "$SERVICE" .

• publish.sh

SERVICE="hysite"
docker tag "$SERVICE" hyston/"$SERVICE":latest
docker push hyston/"$SERVICE":latest 

And after that I have to ssh to linode virtual linux and run third script there:

• run.sh

docker stop $(docker ps -aq)
docker rm $(docker ps -aq)
docker rmi $(docker images -q)
docker run --rm -d -p 80:5000 -v /root/logs:/app/logs hyston/hysite:latest

I think everything here is self is self-explanatory¹ and easy enough to be written by a person, who is started to looking ways to develop simple applications into the cloud. Deployment of new version was a cumbersome, but reliable path of running all three of these scripts together.

There were two things, that I didn’t like about this approach:

• This all totally can be automated. Run first two on git pull into master and hook third one when new image was pushed into docker hub.
• Here I was using linux virtual machine, but with only reason - to run docker on it. On first versions, things were a bit more complicated (I have stored posts in shared with container folder), but now it is just the environment, that can run docker, with is, on my opinion, overkill.

Goal

What I was looking is service, that can take docker file, access to repository, restore, build, publish and create docker image, that will contain latest version, then deploy it into some cloud – and all of this should happen automatically, on new commit in master and without any manual steps. Ideally, it also should beforehand check, that tests are running, builds are building and notify me, if something is wrong.

Dead ends

Because I use only one image, I ignored all solutions, that have “Kubernetes” in name. Dunno, it just scared me a little, I was looking for a simple ways to start. I knew, that there are two major cloud operators: Azure and AWS and for my purposes I decided to stick with azure. They have some new service, called container instances, that is used exactly for what I need – take docker file, build image, run it and don’t ask any more questions.
Besides azure portal, MS have product “devops pipelines”, which purpose is still little unclear for me. I have created pipeline, that took repository, build my project and run it in container instance - but I met a problem.
All posts in this site stored in private GitHub repository and first thing, that hyston.blog is doing on start – is attempt to sign in into GitHub and load all posts. To do this, obviously, I need to store my GitHub credentials. When I build site on local machine, these username/password were stored in web.Production.config, which was gitignored. It is possible to add azure pipeline secrets, that would contain GitHub username and password and pass them into docker environment variables, but then they will be stored as linux environment variables, which is considered as insecure. There is other option to do this, that I’ll describe late, but by then I decided, that is a stop sign for me to use container instance. The other reason, that pushed me away from container instances was this comment in Stackoverflow:

Azure Web App for Containers is targeted at long running stuff (always running) while ACI are aimed at scheduled\burstable\short lived workloads (similar to Azure Functions).
So, despite having already running instance in ACI (without GitHub credentials, though), I was switched to use Azure web app. And the problem started to fall on my head.

Azure Web app

At first, I have decided created webapp with deployment directly code – so I should note care about docker at all. There is “deployment configuration” option in web app, that (if I remember correctly, it is using devops pipelines) should work as “wizard”. For me it refused to login into my GitHub account at first². After a day or two this issue has gone and I was able to create deploy my code directly in couple clicks. But there appears another problem: there is no way to choose, which linux distribution will be used to deploy my code. For GitHub integration, my blog is using LibGit2Sharp, which in turn, using very specific linux git library, that is exist only on Ubuntu LTS. Some details about this issue can be found here. For docker image, I just add ‘-bionic’ to runtime dotnet image and then it runs in ubuntu. But for automatic deployment configuration base image was Debian and my site throwing an exception on trying to use libgit2sharp. At this point, I was frustrated about Azure portal functionality overall and confused with Azure devops pipelines (as I understood, these are two different services, that use different accounts). So I went further into GitHub Actions.

Github actions

There are many ways and possibilities, that are opening, when you are using them. Simplest way is to build dotnet core app directly and deploy binary somewhere. Because I already added build layer to my docker file, I left compilation part there, although I dont like this solution now – wherever I deploy it now, it doesn’t use cache from previous deployments. It is possible to set my own machine as action environment, but I currently don’t have any computer that will run all the time.

Azure KeyVault

I’ll show my deployment script later, but before that I still have one unresolved problem: I do need GitHub credentials in runtime and I still don’t want to store them in open source configuration. Because I stick with azure, I decided to use azure vault. It is overloaded and cumbersome service (as everything with Microsoft), but it ended up well documented (probably, even overdocumented, because I have found several very similar tutorials) and with this tutorial I have added support for it in my code. When I run locally, it take azure credentials from my machine (because I have installed and logged in azure cli) and in docker I have added environment variables, that imported from arguments. This is more secure, than just storing plain credentials in environment, because with azure client id this can be retained only from azure web app. And to pass them into dockerfile, I store them in GitHub secrets, which is inaccessible from view. So, to recap:

1. Azure credentials (client id, tenant id and client secret) stored in GitHub secrets.
2. When GitHub Action is running, build docker image with those secrets as arguments
3. In Dockerfile, they are coverted into environment variables.
4. When in code I create Azure.Security.KeyVault.Secrets.SecretClient it take azure credentials from environment automatically
5. With this client, I request my GitHub username/password from azure secret vault.

It may sound overcomplicated (that was stopped me from using it for a long time), but in hindsight this is most secure solution, as I can see.

Github actions (again)

To finish my GitHub action I needed only one step: deploy builded image. After all issues that I have seen trying to create devops pipelines, this was easy.

1. At first, azure container repository should be created, where actual images are stored. At “access keys” tab I need to create admin user, that can be used for accessing images.
2. Add step azure/docker-login stop into GitHub action to login using these admin credentials
3. After docker build, this image should be pushed to container registry.
4. After image push is done, use azure/webapps-deploy to do actual deploy. Here I need to store another GitHub secret: azure publish profile.
5. Back to azure, I need to get permission to pull image from repository into webapp. This is well described here. ³

Happy end

And that’s basically it. It took me more than one hour to write this post about a thing, that took me almost a month to figure out ⁴. In the end it is just two scripts: Dockerfile, deploy.yaml created azure account and several settings in azure portal and GitHub. I hope, that in a year, when I will forget about all of that and would need to implement similar stuff, this blog post will help me to refresh my memory.

P.s. Have I mentioned, how I hate yaml indent-based structure? 🤯 This is good in theory and keep structure organised, but I have made too many errors during my development adventures.