Intro
With Steve Jobs originally presenting the idea of web apps “that look exactly and behave exactly like native apps” 12 years ago and 4 years since the term “PWA” was coined, How close are we to a webpage that behaves like a native app? First lets take a look what makes an app native:
- Look & feel
- Data access anytime
- Native Apis
I will show you, how you can develop a web application, that just works as well as a native app.
The look & feel
Low effort big result: webmanifest & serviceworker
I assume you are using Angular, but for other frameworks the principles should be the same. To turn your web app into a PWA, it takes 3 basic steps:
Run
ng add @angular/pwainside your project. This will create two files:manifest.webmanifestandngsw-config.jsonand automatically embed them into the index.html.Install the pwa-asset-generator and run it inside your project, like so :
pwa-asset-generator ./assets/icon/favicon.png ./assets/icons -b "#292d3E" -i ./index.html -m ./manifest.webmanifest
This will create all necessary icons and splashscreens for all platforms, taking the favicon and a color code as parameters. It will also update your index.html and webmanifest accordingly.
- Update your
ngsw-config.jsonto specify which resources you want to stay in the cache. Choose your install and update mode. I am usingprefetchat entkraefter.pro, because i want the audios to be available as soon as possible and notlazyload them.
{
"name": "assets",
"installMode": "prefetch",
"updateMode": "prefetch",
"resources": {
"files": [
"/assets/**",
"/assets/mp3/**"
]
}
}
Ionic - Angular just better
Ionic is a web framework that was originally only compatible with Angular, but with the redesign in Ionic 4 to use standardized web components, it is now also available for React, Vue and plain Javascript. Its main goal is to provide an UI toolkit for developing high-quality cross-platform apps for native iOS, Android, and the web—all from a single codebase. Ionic is open source and has been around since 2013, providing web components, that automatically style and behave like the host system their running on. But that is not all, it has many more nice features, like a grid system (so no need for bootstrap) and especially for Angular, out-of-the-box lazy loading, advanced life cycle hooks and more. The Ionic CLI is very helpful and powerful. Check out the docs to get started.
The data - an offline first approach
So far we have create a dumb website that can be installed on 93% on devices of current users browsing the web. Let’s make it dynamic and smart by adding an offline Database with PouchDB and syncing with a remote CouchDB instance. We will also take a look on how authentication and user roles can work with superlogin and nano. Let’s start.
Brief introduction to NoSQL
In a relational database, data is stored in a set of tables that are made of of rows and columns. A Structured Query Language that consists of keywords like SELECT, FROM, WHERE, and JOIN can be used to query these tables for information.
Tables in a relational database have a pre-defined “schema”. A schema defines the structure of the tables, and the type of data that will be stored in them. In MySQL, for example, you might define a “schema” for a table like this:
CREATE TABLE Cars (
id INT(6) AUTO_INCREMENT PRIMARY KEY,
make VARCHAR(30),
model VARCHAR(30),
year INT(6),
purchased DATETIME
)
Unlike a relational database, a NoSQL database has no predefined schema and does not store data using related tables. NoSQL is not one specific thing, but in general a NoSQL database is not relational. CouchDB is a document based NoSQL database. A document in this context is simply a JSON object like this:
{
"_id": 1,
"name": "Max",
"country": "Austria",
"interests": ["Ionic", "IOTA", "Insect Protein"]
}
Introduction to CouchDB & PouchDB
There are many advantages to using CouchDB including the ease of which it can be scaled, and the speed of read and write operations, but the killer feature when it comes to mobile applications is its ability to synchronize between multiple databases. A CouchDB database implements a RESTful API, which means we can interact with it using HTTP methods like GET, PUT, POST, and DELETE. So, if we wanted to read some data from a CouchDB database we might make a GET request to the following URL:
http://someserver.com/mydatabase/_design/posts/_view/by_date_published. When installing CouchDB locally it comes with a web gui called Fauxton, to modify databases.
PouchDB is a CouchDB style database that runs locally on the user’s device. It can be used independently, or it can be used in conjunction with other remote CouchDB databases.
When using the PouchDB library in your Ionic application, you could trigger a sync between a local database (i.e. one running on the users phone) and a remote database (running on a server) with a single line of code:
PouchDB.sync('mydb', 'http://url/to/remote/database');
To provide easy scalability, fast reading and writing, and synchronization, CouchDB prioritizes Partition Tolerance and Availability over Consistency, unlike traditional MySQl Databases. If two users try to update a document, one online and one offline, the updated doc of the offline user will be rejected when he comes back online. CouchDB assigns documents with a “revision” number, which is stored in the _rev field. If the _rev fields match, CouchDB will process the update and increment the _rev number. If the _rev fields do not match, the update will be rejected. What this means for our situation with two simultaneous users, is that whoever syncs their update to the remote database first will “win” and have their update accepted.
The following is an example of a data.service that handles database operations and synchronization to databases the user has access to. For this to work, you need to add the DataService into the constructor of your app.component.ts. The initDatabase function is triggered by the auth.service, when the user logs into the application. We will take a look at this in a second.
import { Injectable } from '@angular/core';
import * as PouchDB from 'pouchdb/dist/pouchdb';
import { UserService } from './user.service';
@Injectable({
providedIn: 'root'
})
export class DataService {
public dbs = null;
private remoteAddress = [];
private remoteName = [];
constructor(private userService: UserService) {}
initDatabase(remote): void {
this.remoteAddress = Object.values(remote.userDBs);
this.remoteName = Object.keys(remote.userDBs);
this.dbs = {};
// save PouchDB instances and remote address, that the user has access to
for (let i = 0; i < this.remoteName.length; i++) {
this.dbs[this.remoteName[i]] =
new PouchDB(this.remoteName[i], {
auto_compaction: true
});
this.dbs[this.remoteName[i]].address = this.remoteAddress[i];
}
this.initRemoteSync();
}
initRemoteSync(): void {
const options = {
live: true,
retry: true,
};
for (const db in this.dbs) {
const dbRemote = this.dbs[db].address;
this.dbs[db].sync(dbRemote, options);
}
}
// Database operations, dbname is provided by service
createDoc(doc, dbname): Promise<any> {
return this.dbs[dbname].post(doc);
}
updateDoc(doc, dbname): Promise<any> {
return this.dbs[dbname].put(doc);
}
deleteDoc(doc, dbname): Promise<any> {
return this.dbs[dbname].remove(doc);
}
}
The remote server.js - authentication and creating users
Our remote server mainly consists of three files: server.js to initialize the server, superlogin.config.js to configure superlogin and superlogin.controller.js to handle user creation and authentication.
Your package.json should look like the following:
{
"name": "my-server",
"version": "1.0.0",
"description": "",
"main": "server.js",
"scripts": {
"start": "node server.js"
},
"dependencies": {
"superlogin": "^0.6.1",
"body-parser": "^1.17.2",
"cors": "^2.8.3",
"couch-pwd": "github:zemirco/couch-pwd",
"del": "^3.0.0",
"express": "^4.15.3",
"https": "^1.0.0",
"method-override": "^2.3.9",
"morgan": "^1.8.2",
"nano": "^8.1.0"
}
}
The server.js should look like this:
const express = require('express');
const https = require('https');
const bodyParser = require('body-parser');
const logger = require('morgan');
const cors = require('cors');
const superloginController = require('./controllers/superlogin.controller.js');
const app = express();
app.use(logger('dev'));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(cors());
superloginController.initSuperLogin(app);
app.listen(process.env.PORT || 8080);
All it does is to initialize our express server. The more interesting part is superlogin. It does all the heavy lifting for us when it come to
- Logging In
- Logging out
- Account Creation
- Validating usernames
- Validating emails
This is not the limit of what superlogin can do. Check out the docs for more information. The full potential can be seen in this sample config.
As PouchDB always syncs at least a part of a whole database, we need to think about how we structure our database to avoid unauthorized access. A database per user setup is the best option. Let’s take a look at our superlogin.config.js:
module.exports = {
dbServer: {
protocol: 'http://',
host: '127.0.0.1:5984',
user: 'admin',
password: 'couchdb',
cloudant: false,
userDB: 'couchdb-users',
couchAuthDB: '_users'
},
security: {
maxFailedLogins: 5,
lockoutTime: 600,
tokenLife: 604800, // one week
loginOnRegistration: false,
defaultRoles: ['user']
},
mailer: {
fromEmail: 'gmail.user@gmail.com',
options: {
service: 'Gmail',
auth: {
user: 'gmail.user@gmail.com',
pass: 'userpass'
}
}
},
userDBs: {
defaultDBs: {
shared: ['shared'],
private: ['private']
}
},
providers: {
local: true
},
userModel: {
whitelist: ['isAdmin'],
isAdmin: false,
},
};
In userDBs we specify to which database our user has access to. We define a shared database where any authorized user has access to and a private database, which only our user can access. If we want to have different user roles, let’s say an admin, who has access to all the other users, we need to modify the userModel to create an isAdmin field. Let’s take a look at our superlogin.controller.js to see how we create users and give admins special access.
const nano = require('nano')('http://admin:couchdb@localhost:5984');
const superloginConfig = require('../config/superlogin.config.js');
const SuperLogin = require('superlogin');
module.exports.initSuperLogin = app => {
// Initialize SuperLogin
const superlogin = new SuperLogin(superloginConfig);
// Mount SuperLogin's routes to our app
app.use('/auth', superlogin.router);
// Create superlogin event emitter
superlogin.on('signup', function(userDoc, provider) {
// opts for replication
const opts = {
continuous: true,
create_target: true,
// exclude design documents
selector: {
"_id": {
"$regex": "^(?!_design\/)",
}
}
};
// get private DB name
const regex = /^private\$.+$/;
let privateDB;
for (let dbs in userDoc.personalDBs) {
console.log(dbs)
if (regex.test(dbs)) {
privateDB = dbs;
}
}
// Replicate design documents to private DB from userDoc
nano.db.replicate('user-resources', privateDB).then((body) => {
return nano.db.replication.query(body.id);
}).then((response) => {
// console.log(response);
});
if (userDoc.isAdmin) {
// Replicate AdminDB to AdminUsers
nano.db.replication.enable('admin-database', privateDB, opts).then((body) => {
return nano.db.replication.query(body.id);
}).then((response) => {
// console.log(response);
});
} else {
// Enable replication from userDB to adminDB
nano.db.replication.enable(privateDB, 'admin-database', opts).then((body) => {
return nano.db.replication.query(body.id);
}).then((response) => {
// console.log(response);
});
}
})
}
First we initialize superlogin and mount our routes to the /auth api. Next up we tell superlogin to listen to signup events. We utilise nano to interact with CouchDB and set-up a few replications across our user and admin databases.
In the opts object we define to replicate everything, but design documents. Design documents are special CouchDB documents that help us to filter documents. We the run a for loop over the userDoc to get the name of our private user database. Next up we replicate design documents, that we need in our private database from the database user-resources.
If the userDoc does not have isAdmin field we replicate our privateDB to the admin-database. If the user does have the isAdmin field, the replicate all docs form the admin-database to the private database of our admin user.
That’s it for the server side part. Let’s go back to our Ionic application to finish the auth.service and user.service.
Authentication on the client side
Update src/environments/environment.ts to reflect the following:
export const environment = {
production: false,
};
export const SERVER_ADDRESS = 'http://localhost:8080/';
Our auth.service.ts looks like this:
import { Injectable, NgZone } from '@angular/core';
import { HttpClient, HttpHeaders } from '@angular/common/http';
import { NavController } from '@ionic/angular';
import { UserService } from './user.service';
import { DataService } from './data.service';
import { SERVER_ADDRESS } from '../../environments/environment';
@Injectable({
providedIn: 'root'
})
export class AuthService {
constructor(
private http: HttpClient,
private userService: UserService,
private dataService: DataService,
private navCtrl: NavController,
private zone: NgZone
) {
}
authenticate(credentials) {
return this.http.post(SERVER_ADDRESS + 'auth/login', credentials);
}
logout() {
const headers = new HttpHeaders();
headers.append('Authorization', 'Bearer ' + this.userService.currentUser.token + ':' + this.userService.currentUser.password);
this.http.post(SERVER_ADDRESS + 'auth/logout', {}, { headers }).subscribe((res) => { });
// destroy all databases
for (const db in this.dataService.dbs) {
this.dataService.dbs[db].destroy().then((res) => {
console.log(res);
}
, (err) => {
console.log('could not destroy db');
});
}
this.dataService.dbs = null;
this.userService.saveUserData(null);
this.navCtrl.navigateRoot('/login');
}
register(details) {
return this.http.post(SERVER_ADDRESS + 'auth/register', details);
}
validateUsername(username) {
return this.http.get(SERVER_ADDRESS + 'auth/validate-username/' + username);
}
validateEmail(email) {
const encodedEmail = encodeURIComponent(email);
return this.http.get(SERVER_ADDRESS + 'auth/validate-email/' + encodedEmail);
}
reauthenticate() {
return new Promise((resolve, reject) => {
if (this.dataService.dbs === null) {
this.userService.getUserData().then((userData) => {
if (userData !== null) {
const now = new Date();
const expires = new Date(userData.expires);
if (expires > now) {
this.userService.currentUser = userData;
this.zone.runOutsideAngular(() => {
this.dataService.initDatabase(userData);
});
resolve(true);
} else {
reject(true);
}
} else {
reject(true);
}
});
} else {
resolve(true);
}
});
}
}
We use the basic api routes provided by superlogin for authentication, registration, validating the username and email. The more interesting parts are the logout() and reauthenticate() functions. Because we are synchronizing multiple databases, we have to make sure to destroy all of them with a for loop in the logout() function.
One main feature is, that we want our users to log in automatically if they have previously logged in and they have an unexpired token. Also users should have offline access to the data in the application that syncs when online. The reauthenticate() function checks for a valid token in local storage and does just that. Let’s take a look at our user.service. This will be a short one.
import { Injectable } from '@angular/core';
import { Storage } from '@ionic/storage';
@Injectable({
providedIn: 'root'
})
export class UserService {
public currentUser: any = false;
constructor(public storage: Storage) {}
saveUserData(data): void {
this.currentUser = data;
this.storage.set('UserData', data);
}
getUserData(): Promise<any> {
return this.storage.get('UserData');
}
}
We utilize Ionics Storage function to save and retrieve the userData on login and reauthentication.
That’s it for the data part. Let’s move on to the native apis.
Native Apis
The main browser manufactures have proposed new Apis to utilize hardware and platform access such as Contacts and WebNFC. Till then we can use capacitor, the successor to cordova, to add native functionality (you can also use cordova plugins). You can also compile your front end application with just a few commands into a full featured native app: Install and initialize capacitor…
npm install --save @capacitor/core @capacitor/cli
npx cap init
…and then choose your platform.
npx cap add android
npx cap add ios
npx cap add electron
With entkraefter.pro I used the share api. Test it out on your desktop, android or ios device to see beautiful, native share dialogs.
Bonus: Deployment
To run node.js applications on your server you should use PM2. For the moment I also use AzureDevOps (i plan to move to a self hosted jenkins solution) to build my front end and server applications on commit and publish them automatically onto my VPS.
Thank you for your time, also check out my Nginx performance tutorial.