Aws lambda + api gateway

The road to success is certainly treacherous and unpaved when using AWS’ stuff. The cultural misappropriation of MSFT’s demo-ware is blatant. Yet, once you know the ins and outs, you can get stuff done.

So I had a case of a bunch of apis that would be internally used, not critical and would make no sense to provision a VM for that. To be more specific, it is used to collect perf tests results and then to display them in different ways.

Best tool for the job: serveless.

AWS lambda 

Amazon’s offering where you upload code and pay as it is used. Note though, that your code isn’t automatically exposed to the outside world.

AWS API Gateway

Bells and whistles to expose an endpoint to the outside world, secured or not, and mapping the incoming request to inside AWS (or another endpoint) including a lambda function.

First, use Apex

Apex will save you a lot of time deploying functions, and it’s convention on how to organize things (in folders, cascading config, etc) is a time saver.

Testing the functions

When testing within the aws console, keep an eye on the cloud watch for the log stream. Define a test input and run. All good? Time to expose it.

Exposing the endpoints

Two ways to do that:

  • Within the function, click on Triggers and add a API Gateway trigger.
  • Alternatively, go to the API Gateway console and define the resources and methods, and then map the resources to the lambda functions.

This turned out to be the tricky part in my short experience. Put it this way: lambda functions dont know about HTTP, and that’s ok. So on the API Gateway console you map parts of your http into the json your function expects.

When clicking on a http method, you’ll see Method Request -> Integration Request -> Lambda -> Integration Response -> Method Response.

Let’s break this down:

  • Method Request: the incoming http part of the equation. Here you have the opportunity to declare query strings params, headers and models (if you want document your api and|or create a client for it) and their semantics.
  • Integration Request: this is key. Lambda doesnt know about http, so here you have the opportunity to get info from the http request, and map into the json your function expects.
  • Integration Response: same thing on the opposite direction
  • Method Response: ditto

To use mapping, disable the “request body passthrough” and add a content type. Note, this will only work for POST, PUT, etc, in other words, requests with a body, thus where Content-Type is applicable.

Add a content type (very likely to be application/json).

Mapping is done by using a Velocity template. Check this out as an example to collect everything. In my case, I just wanted a param from the query string, so I used:

{ “myparamname”: “$input.params(‘querystringkeyname’)” }

Ready to test? You can test internally by using the gateway “Test” function. but if you want to use Postman or something else, then:

Deploy your api

You can create different “stages” (test, staging, production) and deploy api there. Once there, the urls are fixed. Changes to mapping and other configuration will require a re-deploy.

Annoyances

I’ve bumped into situations where I couldn’t delete methods or resources, even after deploying the new trimmed one, deleted ops would still show up.

The solution? Deleting the whole deployed api. The problem with that is that a new deploy comes with a new URL. It’s ok for us since we only use this internally, but in production this could be a huge headache.

Update

See Damien’s notes on the security issues with json mapping

Erlang from the perspective of a dev used to OOP – Part 1

So as many people out there I’ve also heard that Erlang was awesome, that all distributed computing problems were solved first by Erlang 30 years ago and so on. Decided to learn it. Surprisingly it’s not hard, and there’s an obvious reason for that.

Splitting hairs

First things first, Erlang as a name is ambiguous. It may refer to two things: the language or the runtime/VM (also known as BEAM). As in .net and java, there are  many languages targeting the Erlang VM, the most popular these days being elixir which combines Ruby syntax with some interesting meta programming capabilities.

That said, before jumping into elixir or something else, you should take some time to get acquainted with Erlang – the language. That’s what I did.

Concept count

To make this venture easier get used to this: you won’t find generics. Actually, let’s go further: no types. Well, no types that you can create, derive, subclass. And no objects, or to be more precise, no instances. Yeah, a functional language without OOP compromises. You still reading this?

Yeah, I know it sounds limiting but sometimes constraints are liberating.

The concept count of Erlang is minimal, which makes it extremely easy to learn it. In sum:

  • Atoms: you know, symbols that can be compared, are always “the same”. In Erlang (language) these are always expressed in lower case like true or false or exchange.
  • Numbers: you know them
  • Variables: which should start with Upper case like MyList. Oh, variables is a bad name since you can’t really change them.
  • Tuples: an aggregation of values like { one, 2, Three }
  • Lists: expressed as [ 1, 2, 3 ]
  • Records: which are more a language trick to make some things easier. In reality they are just tuples of name(atom) and a value.
  • Strings (or binary lists). Awkward, but I’ll get to them later

Unless I forgot something, those are the “things” you have to deal regularly when writing Erlang code.

Now IMHO the language syntax isn’t the most beautiful, terse, poetic you may find. But it’s practical and you get used to easily.

On future posts I intend to cover the following topics:

  • Modules and writing your first code
  • The shell + observer + debugger
  • Processes (the closest thing you get to an “instance” of something)
  • Pattern matching
  • Some OTP

YMMV and I’m not an Erlang expert. But after writing a fake security exchange that we use it here to run our integration tests and perf tests, I’m comfortable enough writing about it.

Your thoughts?

Rethinking my dev stack

Against all odds, I’m still a big fan of .net. I like their core API, some of the surrounding stack, I *love* F#, the tooling (VS + Resharper). It’s a quite productive place to be.

However it’s undeniable the amount of innovation happening elsewhere, and the playing catch up that DevDiv’s doing. Some things we can mimic in .net (like a mvc framework based on Rails), but try to spawn/deploy lightweight processes the same way erlang does.

There’s a firm and passionate community around .net hanging there, witnessing their open source offering being abducted by the mothership. There are also communities around different stacks that seems to play a more important role and being appreciated for it.

For example, the way node.js was designed to support modules, anticipating that node itself is a lightweight kernel, and it’s usefulness is limited only by the imagination of its community. Js may have its problems (dont I know it!) but since it’s fully composable, you can define light contracts and unleash a world of plugins and combinations of how they interact with each other. Express seems to be doing that very well.

On the cost side of things, hosting a Windows solution on aws or azure is about double the price of their linux counterpart. The question is, does the productivity gain of the .net stack + tooling (which isn’t free) outweigh this operational cost that will last well, the lifetime of your app? I’m starting to doubt that.

So I took this holiday to experiment with node + aws + typescript. While there’s some rough edges, the learning curve is practically flat. The fear I always had is the growth in complexity of an app and javascript don’t mix well. But typescript seems to mitigate that gracefully. Another mitigation is the event emitters/listeners that node seems to encourage. Nothing new there, but a good pattern nonetheless.

The dev-polyglot movement seems to make more sense everyday. Be tied to a single stack is a liability, it can go up or down in demand – and Microsoft had better days. So read about http://en.wikipedia.org/wiki/Sunk_costs and go learn something new.