← back to the blog


Instal keystone.js using Mongodb Atlas as backend

Posted
on October 15th, 2018 in Javascript, nodejs by Chanoch

Mlab - the defaul option

I have always previously used mlab for my small time mongo based nodejs systems. The solution was always good enough and their free sandbox services were nicely performant. In addition, the web admin is nice and pretty with plenty of useful features.

Why Mongodb Atlas

Earlier this year, however, when I was looking at a production deployment for a paying customer mlab somehow didn't fit the bill. I can't remember the details - but I would have looked at security, cost and cost scalability. I suspect cost was the main factor. They offer a free tier host in Germany or the US. They have a London host but you have to pay for that.

In any case, I found mongodb's own saas offering Atlas available here MongoDB Atlas (disclosure: I get coffee money if you sign up from that link). The online portal is in MVP stage with a few rough edges, but the cost is good, there is a free offering in a shared cluster I've been happily using it for three months.

Comments on the service itself I will leave to a separate post. For now I wanted to show some of the key steps for setting up keystone using Atlas rather than a local mongodb server.

What is keystonejs?

I chose keystone as it is developed in nodejs and react (on the admin portal) and uses mongodb as the backend - my current working stack. It also uses jquery - the wordpress of the frontend js world - but it doesn't seem to get in the way too much.

keystonejs is definitely a developer's platform - you quite quickly need to get into the guts of it in order to customise it and the theming support feels a bit immature. On the other hand, if you know expressjs and node, it is very convenient and easy to figure out - no need to read the documentation.

Setting up keystone - assume a local unauthenticated mongodb instance

When I first set up keystone, it wasn't apparent that the database connection assumed a local install of mongo with no authentication. When it first starts, keystone creates the database and collections in mongo and these are then required to be able to start it again so it is important to change the default configuration before starting it for the first time.

Start by experimenting

I started by consulting building a few vanilla sites to confirm the options and then scrapping them straight away - using yeoman allows you to do that quite easily. Just rm -rf the folder and start again. Once I had an initial build working, I consulted the article from mongodb about setting up keystone with Atlas

Installing keystone with Yeoman

I started by using Yeoman to configure a vanilla site - pretty much accepting all the defaults.

yo keystone

This provides mostly default options but the following have affects:

  1. Name of your project - this will define the name of your database and project folder
  2. Template language - default is pug but I like nunjucks. Most people seem to use handlebars so there's more help for it as well as stackoverlow copy/paste development available
  3. CSS preprocessor - sass support on ubuntu seems to be tricky as it required python and make to work. I tend to choose stylus and I believe there is a js native sass implementation
  4. The contact form requests mailgun credentials and emailing hasn't worked for me so far. The contact requests are stored in the admin console (/keystone) so ok
  5. You will need to provide an email address for the initial admin user
  6. note: the admin user's password will be stored in plain text on your file system so choose a throwaway password for safety's sake. It is stored encrypted in mongo so not the end of the world - make sure you change it later. the default is crazy (user@keystonejs.com) so shouldn't really be offered as a default
  7. keystonejs uses cloudinary (link opens in new window) to store digital assets. Cloudinary offer 10GB and 300,000 max images with 20GB transfer free each month so probably enough for your personal blog. The cheapest paid option is £60-70/month so not a deal breaker for businesses. Pretty impressive offering including video processing and image manipulation but not enough experience with them to recommend them as yet. I am certainly going to give them a go.

BEFORE YOU START KEYSTONE

Before modifying your code base, it's a great idea to snapshot the build in git so you can get back to it later if you want to unbreak something you customised.

If you want to set up a connection to Atlas then before you follow their instructions, register, login and set up your Atlas config.

Setting up the database

Keystone will create your database for you if you ask but you would need to give it your atlas admin (superuser). Maybe you could go ahead and do that initially then swap it out later but the temptation would be not to bother so I went ahead and create the db myself.

Databases in atlas are created in the collections page. The name of the database needs to be the same as your blog name - I think there are some translations to avoid reserved characters in mongo (such as a dot/period). Spaces are replaced with dashes so 'My Site' becomes 'my-site' as your db name.

Configure a user

Database users are configured at the cluster level so go back up a level. It would be a bad idea to use your superadmin user in mongo for your blog so go ahead and set up a new user for the blog. Keystone needs DBAdmin and Read/Write and give the above, i would never use the only one of their three default options that would work (Atlas admin). If you hit the Show Advanced Options link below you will then be able to target a specific database.

Add a username and password (avoid special characters in the password - it'll only cause you issues later. Try a generated alphanumeric password created by bitwarden) the two roles for your blog database then Add User

While you are here, it would be a good idea to configure IP whitelisting for your server and your dev machine. This is where a VPN solution with a static IP option (I use purevpn's - nice people and support is excellent for Ubuntu and OSX) would be a good idea so you get a consistent IP address as well as better transit security for your network data.

Later on it would be a good idea to delete the DBAdmin role from this user and check if that breaks keystonejs. Reduce the attack surface area - it's obviously going to need read write but it should need to be able to create and drop collections just to maintain content.

Configuring your database connection

To configure Atlas for keystone, you need to edit the .env configuration. Do not save this into git as it is going to contain a plain text database user password. Alternatively you can provide the connection string in the terminal instead.

The connection string takes the following format (this is a single continuous line but I've cut it up for readability The space-backslashes-line return are ignored in Ubuntu so the computer sees a single continuous string:

MONGO_URI="mongodb://db_username:db_password@ \
cluster0-node1.mongodb.net:27017, \
cluster0-node2.mongodb.net:27017, \
cluster0-node3.mongodb.net:27017 \
/database-name?retryWrites=true&ssl=true&authSource=admin"

 

This URL is not the one which Atlas provide you with. As of April 18 they have started offering a DNS based connection string which allows you to dynamically enroll additional nodes and change the node members of a cluster but the mongo drive which keystone uses doesn't support mongo's new mongodb+srv:// prefix so we will need to list the cluster node members separated by commas using the old URI format for mongo clusters. The database name is at the end followed by some options given by the connection string you get if you hit the 'connect' button in Atlas in your cluster.

You can see the names of the cluster members if you click your cluster (probably cluster0). The ports for your nodes all use the standard mongo port 27017

At this point you are ready to start up keystone.

Configuring mailgun

To set up mailgun you will need to register a domain (e.g. m.mydomain.co.uk and configure DNS for it accordingly to ensure the  SPF and domain authentication TXT records) Once the domain is validated, you will get your keys.

The following two settings configure the mailgun secret and host for your mailgun integration.

MAILGUN_API_KEY={your api key}
MAILGUN_DOMAIN={m.yourdomain.com}

 

My experience is that you MUST set up your domain in the US region. Trying to specify a domain that is set up in the eu region fails with a 'domain not found' error.

Starting keystone

As the Yeoman generator suggests, you can cd into your directory and start keystone by typing in node keystone. I don't know why they don't suggest using the more standard npm start. The exception message I got when I was messing about were pretty helpful. It will also put out a console message about your mail gun settings if you didn't set these up.

At this point, if your user is set up correct you will see your blog at http://localhost:3000

CHANGE YOUR ADMIN PASSWORD

You can log in to your blog by clicking a relevant button or going to http://localhost:3000/keystone

Deploying your blog

This one is for another day. You should be able to do it by logging in to your server with SSH and cloning your git repo then:

cd My-site

Create your .env file with the various settings copied from your local machine. Add in a NODE_ENV=production variable to optimise the install, allow keystone caching and improve start up times.

npm i && npm start

And you are good to go. To keep the service persistent between server restarts and process exits, have a look at pm2 (opens in new window), an awesome process manager for node by keymetrics which I use and will use extensively until I can look at docker again.

Other points

Did you change your admin password? Do it now.

You can change the port as usual with express apps by setting a PORT= env variable at the terminal and/or the in the .env file.

It will also have set up collections in your Atlas account with some default values.

If you look in the your updates/ folder, it contains scripts to set up data. Any files in there will be processed and a record of the file having being processed set in your database. This can be a useful way to import data into your blog. This is the way I would migrate blog entries from another blog engine. There are some notes in the 0.0.1-admin.js file to get your started.

Photo by Artem Bali on Unsplash
d