Kong api gateway. Getting started

Automating Kong api gateway setup with Terraform

This post is part of the tutorial series about using Kong api gateway product in your technology stack.

Kong is an open source product based on the Nginx server and OpenResty framework. Both of these underlying technologies provide Kong with the high transaction throughput and very low memory footprint foundation.

The goal of this tutorial is to get started with Kong api gateway and build secure public api endpoint.

What will we build?

By the end of this tutorial you will have working Kong setup with new service and route for public api. With api key security implemented.

Prerequisites

Access to Debian Linux server. Any server works, you can use VirtualBox locally or one of the cloud platforms (I'm using Digital Ocean in this tutorial - for the new signups they apply lots of free credits).
Docker installed (Mac or Windows).
Api testing tool of your choice. We will be using Insomnia Rest Client in this post.

1. Provisioning Virtual Host

For this tutorial, we will use temporarily Debian Linux VM instance on Digital Ocean. It allows to simply create and delete virtual machine droplets for testing and the proof of concept work.

Once we have the new server up and running, we use ssh to connect and prepare it for installing Kong.

Run the following shell commands to get started

apt-get update
apt-get install -y apt-transport-https curl lsb-core

# optional firewall rules
# we will need external ports 8000 and 8001
# for this tutorial
apt-get -y install ufw
ufw status verbose
ufw default deny incoming
ufw default allow outgoing
ufw allow ssh
ufw allow 22
ufw allow 8000
ufw allow 8001
yes | ufw enable

2. Install Datastore

Next, we will setup the datastore.

Kong api gateway can run as a stand alone application, without any datastore.

In which case it stores all the configuration and setup in memory.

For a more robust setup, Kong supports 2 types of databases. PostgreSQL and Cassandra.

If data store is configured, Kong persists all the api configuration and setup there.

We are going to use PostgreSQL for the purpose of this tutorial. Run the following via ssh connection

install postgresql
# setup kong user and kong database
apt-get install -y postgresql postgresql-contrib
sudo su - postgres -c "createuser -s kong"
sudo -u postgres psql -c "ALTER USER kong WITH PASSWORD 'kong';"
sudo su - postgres -c "createdb kong"

3. Install Kong api gateway

We will use apt-get package manager to install Kong api gateway official Debian package.

# configure apt-get with Debian package details
# and install
echo "deb https://kong.bintray.com/kong-deb `lsb_release -sc` main" | sudo tee -a /etc/apt/sources.list"
curl -o bintray.key https://bintray.com/user/downloadSubjectPublicKey?username=bintray
sudo apt-key add bintray.key
sudo apt-get update
sudo apt-get install -y kong

4. Create Kong configuration

Next, we have to create Kong configuration.

Kong supports file based configuration which is maintained in kong.conf. This file is read once kong start or kong prepare commands are used.

Under the hood, based on this configuration file, Kong generates Nginx configurations and reloads them.

After we installed Kong Debian package in the previous step, the home directory was created at /etc/kong. Navigate there and you'll find it contains the example configuration file.

Copy kong.conf.default to kong.conf

Now, you have to un-comment and edit the lines in the database section. We use the PostgreSQL details we provisioned in the previous step.

pg_user = kong
pg_password = kong
pg_database = kong

5. Start Kong api gateway

We can now bootstrap and start Kong application.

Bootstrap step is needed for Kong to connect to database and migrate the schema. Go to config file directory and run the following

kong migrations bootstrap [-c kong.conf]

Now you can start the Kong api gateway

kong start [-c kong.conf]

6. Test Kong Management apis

Kong Api gateway provides the set of management apis to configure and manage api gateway during the runtime.

Management apis provide us an array of useful data, let's try few API requests.

curl localhost:8001/status | python -m json.tool

{
    "database": {
        "reachable": true
    },
    "server": {
        "connections_accepted": 6,
        "connections_active": 1,
        "connections_handled": 6,
        "connections_reading": 0,
        "connections_waiting": 0,
        "connections_writing": 1,
        "total_requests": 7
    }
}

curl localhost:8001/services | python -m json.tool

{
    "data": [],
    "next": null
}

curl localhost:8001/routes | python -m json.tool

{
    "data": [],
    "next": null
}

There are no services or routes configured yet.

We will use Kong management apis to setup our first public api service and add security plugins in the next steps.

Note: Kong management apis are very powerful, and allow full control of the setup. Make sure you do not expose these api endpoints publicly for your live projects.

8. Create public api on Kong

Kong api gateway has the concept of services and routes.

Services define the connection to the backend service. Routes manage the incoming request configurations.

Let's create the first Kong service which forwards API requests to httpbin.org backend.

curl -v -XPOST \
  http://localhost:8001/services \
  -H 'Content-Type: application/json' \
  -d '{"name":"api1","retries":1,"host":"httpbin.org"}'

{
  "host": "httpbin.org",
  "created_at": 1556618729,
  "connect_timeout": 60000,
  "id": "873c5cd8-26ff-4dd1-a3a4-e0c48ad72f18",
  "protocol": "http",
  "name": "api1",
  "read_timeout": 60000,
  "port": 80,
  "path": null,
  "updated_at": 1556618729,
  "retries": 1,
  "write_timeout": 60000,
  "tags": null
}

Next, we need to create the route for incoming requests.

For the service id field, we use the service id received in previous step.

curl -XPOST \
  http://localhost:8001/routes \
  -H 'Content-Type: application/json' \
  -d '{"name":"api1-route", "protocols": ["http", "https"], "methods": ["GET"],"hosts": ["popularowl.com"], "paths": ["/api1"], "strip_path": true, "preserve_host": false, "service": {"id":"873c5cd8-26ff-4dd1-a3a4-e0c48ad72f18"} }'

{
	"created_at": 1556619873,
	"destinations": null,
	"hosts": [
	    "popularowl.com"
	],
	"id": "32cacdc6-6a03-4878-94f5-e2dcca6936ed",
	"methods": [
	    "GET"
	],
	"name": "api1-route",
	"paths": [
	    "/api1"
	],
	"preserve_host": false,
	"protocols": [
	    "http",
	    "https"
	],
	"regex_priority": 0,
	"service": {
	    "id": "873c5cd8-26ff-4dd1-a3a4-e0c48ad72f18"
	},
	"snis": null,
	"sources": null,
	"strip_path": true,
	"tags": null,
	"updated_at": 1556619873
}

With both service and route successfully created, we can now test our public api on Kong. Remember, Kong public apis are by default accessible on port 8000.

We will use Insomnia http client.

Our API is up and running!

9. Add Api key security

Last step in this tutorial is to add api key protection for our public api.

Note: it is not a best api security practice to rely on api keys alone. They can be easily extracted from api client applications and abused. Consider OAuth 2.0 or OpenID Connect flows for advanced API security

We cover more advanced Kong public api security topics in other tutorials.

In order to add api key validation, we will use core Kong plugin key-auth.

Enable the plugin for our Kong API gateway

curl -i -XPOST \
  http://localhost:8001/services/api1/plugins/ \
  -H 'Content-Type: application/json' \
  -d '{"name":"key-auth"}'

...
"config": {
    "key_names": [
      "apikey"
    ],
    "run_on_preflight": true,
    "anonymous": null,
    "hide_credentials": false,
    "key_in_body": false
  },
...

Now, once you repeat the previous request to our public api you should get http 401 response.

10. Create Kong api key

Next, we need to generate a valid api key.

In order to create an api key, we will use Kong management apis to create a consumer object. After consumer is created, we will assign test api key to this consumer.

curl -i -XPOST \
  http://localhost:8001/consumers/ \
  -H 'Content-Type: application/json' \
  -d '{"username":"popularowl"}'

{
  "created_at": 1556626657,
  "id": "813b9b03-6271-40ba-a88f-b6bc14e45cf4",
  "tags": null,
  "custom_id": null,
  "username": "popularowl"
}

curl -i -XPOST \
  http://localhost:8001/consumers/popularowl/key-auth \
  -H 'Content-Type: application/json' \
  -d '{"key":"test1234"}'

{
  "created_at": 1556626750,
  "id": "30a06e45-1da0-453d-868f-acd56007fa0b",
  "key": "test1234",
  "consumer": {
    "id": "813b9b03-6271-40ba-a88f-b6bc14e45cf4"
  }
}

After the consumer and the key are created Kong by default will allow this consumer key the access to all apis. Our request is successful again.

Notice the additional information passed back in the echo api response?

Response now contains consumer id and consumer username.

Which means Kong has passed the details of the consumer to the backend service.

Summary

In this tutorial we have covered the most important steps to get started with using Kong api platform.

Our Kong tutorials continue with automating all of the above setup steps. This way Kong setup and configuration can be maintained as code.

Other tutorial parts provide practical examples of advanced api security with Kong. Managing multiple apis of several internal microservices. Monitoring and connecting Kong to UI dashboards.