Spring and Groovy

Grails is the standard for rapid enterprise application development, but sometimes you need to fall back on plain old Java/Spring. However, you don’t need to sacrifice the productivity increase of groovy. You can use groovy classes and code interchangeably with your java in your Spring project.

Here we are using gradle, which is arguably more powerful (scripting language vs XML) and faster.

We don’t even need to install Groovy – gradle handles this for us.

Simple add the following to your spring projects build.gradle:

plugins {
    id 'groovy'
}
dependencies {
    compile 'org.codehaus.groovy:groovy-all:2.3.11'
}

Here is an example of a basic Spring boot web project’s build.gradle file which was generated with Intellij’s Spring Initializr feature:

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:2.1.6.RELEASE")
    }
}

plugins {
    id 'groovy'
    id 'java'
}

apply plugin: 'idea'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'

bootJar {
    baseName = 'gs-spring-boot'
    version =  '0.1.0'
}

sourceCompatibility = 1.8
targetCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    compile("org.springframework.boot:spring-boot-starter-web")
    compile 'org.codehaus.groovy:groovy-all:2.3.11'
    testCompile group: 'junit', name: 'junit', version: '4.12'
}

This will generate you a groovy folder under src/main. So you will have:

<projectDir>/src/main/groovy/
<projectDir>/src/main/java/

Now you can create your spring components either as groovy or java.

e.g. create src/main/groovy/hello/MyGroovyController.groovy

package hello
import org.springframework.web.bind.annotation.RequestMapping
import org.springframework.web.bind.annotation.RestController

@RestController
class MyGroovyController  {
    @RequestMapping("/")
    public String index() {
        "Hello from Groovy!"
    }
}

And create a java controller in the file src/main/java/hello/MyJavaController.java

package hello;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.bind.annotation.RequestMapping;

@RestController
public class HelloController {
    @RequestMapping("/java")
    public String index() {
        return "Hello from Java!";
    }
}

For completeness, here is the Application class which makes it all work (src/main/java/hello/Application.java)

package hello;

import java.util.Arrays;

import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;

@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

    @Bean
    public CommandLineRunner commandLineRunner(ApplicationContext ctx) 
        {
        return args -> {
            String[] beanNames = ctx.getBeanDefinitionNames();
            Arrays.sort(beanNames);
            for (String beanName : beanNames) {
                System.out.println(beanName);
            }
        };
    }
}

To run your app, enter the following command:

./gradlew build && java -jar build/libs/gs-spring-boot-0.1.0.jar

Alternatively, you can run the app from the IDE. In intellij, open the gradle tab on the right, open up Tasks, and select application/bootRun.

Now you can hit http://localhost:8080 with your browser and see “Hello from Groovy!” or hit http://localhost:8080/java and see “Hello from Java!”

Advertisements

Exporting excel with grails 3

Usually your controllers will respond with list of data to be show on the screen as paginated tables. However, sometimes your uses will want to download the complete dataset as an excel spreadsheet. This is remarkably easy thanks to the Excel-export plugin here.

Firstly, add this to your dependencies block in your top grails.

dependencies {
    compile 'org.grails.plugins:excel-export:2.1'
}

Let us say you have a domain object books, and a controller with a a search method when the user can select some search criteria:

   def search(Integer max) {
        params.max = Math.min(max ?: 10, 100)
        def books = booklService.find(params)
        respond books, view:"search", model:[books:books, author:params.author, params.title ]
    }

And a gsp something like this:

<g:form name="search" url="[controller:'book',action:'search']"&gt;
        <div class="form-row"&gt;
            <div class="col-md-5"&gt;
                <input type="text" class="form-control" size="20" name="author" value="${author}" placeholder="author" /&gt;
            </div&gt;
            <div class="col-md-5"&gt;
                <input type="text" class="form-control" size="20"  name="title" value="${title}" placeholder="title" /&gt;
            </div&gt;
            <div class="col-md-2"&gt;
                <g:actionSubmit value="Search" action="search" /&gt;
            </div&gt;
        </div&gt;
</g:form&gt;

        <div id="list-book" class="content scaffold-book" role="main"&gt;
            <h1&gt;<g:message code="default.list.label" args="[entityName]" /&gt;</h1&gt;
            <g:if test="${flash.message}"&gt;
                <div class="message" role="status"&gt;${flash.message}</div&gt;
            </g:if&gt;
            <f:table collection="${bookList}"/&gt;
            <div class="pagination"&gt;
                <g:paginate total="${bookCount ?: 0}" params="[author:params.author, title:params.title]"/&gt;
            </div&gt;
        </div&gt;

Now just add a download button on the search form:

            <div class="col-md-2"&gt;
                <g:actionSubmit value="Search" action="search" /&gt;
                <g:actionSubmit value="Download" action="download" /&gt;
            </div&gt;

And add the download method in your controller:

def download() {
    params.max = 1000 // @TODO put this in a setting.
    def books = booklService.find(params)
    def headers = ['Created','Author', 'title']
    def withProperties = ['dateCreated','author.name', 'title]
    new WebXlsxExporter().with {
        setResponseHeaders(response)
        fillHeader(headers)
        add(books, withProperties)
        save(response.outputStream)
    }
}

That’s it. It will download a nice excel file with todays date and time as the file name.

If you want to get fancy, you can manipulate sheets, rows, columns and cells directly using the underlying Apache POI libs. E.g. if you want to add a sum formula at the bottom of a row, you can do something like this:

def download() {
    params.max = 1000 // @TODO put this in a setting.
    def books = booklService.find(params)
    def headers = ['Created','Author', 'title']
    def withProperties = ['dateCreated','author.name', 'title]
  
    int last = books.getTotalCount() + 1 // for header
    def exporter = new WebXlsxExporter()
    exporter.setResponseHeaders(response)
    exporter.fillHeader(headers)
    exporter.add(deals, withProperties)
    def totalsRow = exporter.getSheet().createRow(last+1)
    def cell = totalsRow.createCell(5)
    cell.setCellFormula("SUM(F2:F" + last +")")
    exporter.save(response.outputStream)
}

grails 3: custom validators and custom error messages

Adding custom validators to grails domain objects is easy. Consider the following domain class:

class Transaction {
    Account account
    BigDecimal debit = 0
    BigDecimal credit = 0

    static constraints = {
        debit nullable: false, min: 0.0, validator: {val, obj -> if ((val == 0.0 && obj.credit == 0.0) || (val > 0.0 && obj.credit > 0.0)) return "transaction.equal" else return true}
        credit nullable: false, min: 0.0
}

This shows how the valiator closure is checking that either credit is not zero or debit is not zero:

validator: {val, obj -> 
     if ((val == 0.0 && obj.credit == 0.0) 
         || (val > 0.0 && obj.credit > 0.0)) 
         return "transaction.equal" 
         else return true
}

The error message to be displayed to the user if the constraint fails will be looked up the following file:

grails-app/i18n/messages.properties

Edit this file and add your message property:

transaction.equal="Either debit or credit must be > zero, but not both"

Grails 3: using enums with Domain Classes

It is common to need somethingStatus or somethingType in your domain classes. e.g. Account.accountStatus might be “open”, “closed”, “suspended” etc.

There are two ways to implement this:

  1. using a separate domain class
  2. using a string or int enum.

If the status has other properties, then use a separate domain class. E.g.

Account.groovy:

class Account {
  User user
  Currency currency
  AccountStatus accountStatus
}

AccountStatus.groovy:

Class AccountStatus {
   String name
   boolean canLogin
}

However, in simple cases, you can use an enum:

Account.groovy:

class Account {
  User user
  Currency currency
  AccountStatus accountStatus
}

enum AccountStatus {
   OPEN,
   CLOSED,
   SUSPENDED
}

Here is an example of usage:

Bootstrap.groovy:


def account1 = new Account(user: user1, currency: gbp, accountStatus: AccountStatus.OPEN).save(failOnError: true)

Grails 3.3. How to fix rounding and truncation of BigDecimal in Fields plugin with scale of more than 3.

If you requite more than 3 decimal places, e.g. to support display crypto currencies (ETH has 18 dp, BTC has 8), and use any of the scaffolded views or fields plugins such as the excellent f:table, f:display etc, you will notice that values of say “1.123456” are shown as “1.12”. Worse still, if you want to update the object using scaffolded views or the fields, even if you don’t change the value, it will be overwritten in the database with the truncated value.

The first step to fixing this is to override the widget used to display either all BigDecimals, or just the specific controller field, or the specific domain object (its your choice).

The documentation to refer to is: https://grails-fields-plugin.github.io/grails-fields/latest/guide/index.html

Lets say this is your domain object:

package me
class Account {
    User user
    BigDecimal balance = 0
    Currency currency
    static constraints = {
        balance nullable: false, scale: 18, precision: 50
    }
}

to override the display of all BigDecimals, create the file:

grails-app/views/_fields/bigDecimal/_displayWidget.gsp

The file should contain something like this:

${value.stripTrailingZeros().toPlainString()} 

Now you should see “1.123456”.

Note:

  1. If you don’t stripTrailing zeros, you will see “1.123456000000000000” if you have scale set to 18 for example.
  2. You can also override the display for a specific field or specific controller, rather than all big decimals (see docs).

Trezor vs Ledger HW wallet for business

There are several comparisons of the trezor and Ledger HW and their corresponding SW wallets. These mainly focus on the end user usability and number of coins supported.

However, when it comes to business, who want to support many customers, and for developers who want to integrate the HW wallets into existing commerce or financial systems, the picture is very different.

The conclusion is that Trezor is far better suited to business than the Ledger, but neither are perfect.

Note, here we don’t make the distinction between Ledger Nano S and Ledger Blue, and likewise between Trezor one and Trezor T, because the later models are basically the same as the former just with larger screen and a few extras.

Architecture

Trezor has everything built into the firmware.  The 500 or so coins it supports are always there and always avialble.  No installation or configuration is required. When Trezor add more coin support, you just upgrade the firmware (which takes less than one minute) and you are done.

Out of the box, the ledger has literally nothing on it, it cant do anything.  You need to install apps using their SW wallet.  Each coin is an app. So there is an app for BTC, one for ETH etc.  You can only install a small number of apps at the same time (something like 4 on the nano and 18 on the blue depending on app size).  If you want to do transactions in more than the allowed number of apps, you have to uninstall and reinstall.

When you upgrade the Ledger, you have to remove all the apps, then reinstall them.

APIs

Trezor has a single unified API which is coin agnostic – e.g. you can generate addresses for any coin with one call, and you can sign transactions for any coin.  The coin (e.g. Bitcoin or Ethereum) is just a parameter.

There are several ways to access the Trezor API. there is a python based command line tool, which allows you to do pretty much anything with a bash script or command line (such as generating 1000 addresses for a given path).  There is also the bridge, which allows web apps to access the Hw through the browser.  This then requires no SW to be installed on the users machine to

Ledger doesn’t have an API as such.  In theory, each app can provide its own API. There is apparently a beta API for the BTC app.  But there is nothing to say that other apps (aka coins) will get APIs, or if they will be unified in any way.

Compatibility

Interestingly, you can restore your 24 word seed on either device.  So they are largely compatible in this regard.  The only issue is that the Ledger uses a different Path for Ethereum than the Trezor by default, so you will need to use a wallet which allows the path to be configured (such as Electrum).

Labels

Labelling accounts and addresses is important for avoiding mistakes and for reconciliation/accounting.  e.g. if you have 10 addresses, you might label them with the 10 customer names you have given them to. Or you might label them for different invoices.

Trezor has an interestingly solution for labels. They dont want to store them on the device, as they will be lost if you lose the Trezor and need to restore it from the Seed. So instead the store the labels in an encrypted file in dropbox. The nice part of this is that other users (in a company) can use their dropbox to securely share the labels between administrators.

Ledger supports labels for accounts, but doesn’t support addresses for accounts so these cannot be labeled.

Trezor can label individual addresses.

Discovery gap limit, accounting, and client funds segregation.

If you have say 20 customers, and you want to invoice each, you would want to give each customer their own account in the form of an address.  This allows proper client fund segregation, easier audit and accounting.  Even better, an account per invoice.  Additionally, you would want to label each account with its function.  At a glance, you can see the balance and transactions on any account.

This should be trivial to do with the HD wallet, as it supports unlimited addresses for a given account path. However, due to the limited SW implementations of the discovery gap, it is not.

Ledger only allow one address to be generated at a time, and only for one account.  It ignores one of the main features of the HD wallet – the ability to generate many addresses, with increasing indexes, for a given account.  It is difficult to compare the ledger wallet to any other wallet, as its feature set is so limited.

Ledger_Live

Above we see an “account” called trstbtc1.  It has one address, which corresponds to index zero. You cannot see this address.

 

You cannot add a second address until there is a transaction on the first.  If you have two customers, and want to give each a separate address for payment, you cant.  You would have to give the first customer an address, wait for them to pay, then the SW will allow you generate the second address.   This goes for their “accounts” also.  The work around I see clients doing every day is the “loading hack“. They generate an addresss, transfer 1 satoshi (or equivalent coin) in to it, wait till it is confirmed (1 hour in the case of BTC), then they can generate a second address.  Next they remove the 1 satoshi, so the customer who gets that address given to him doesn’t get worked that it has a balance (so is a “used” account), and do the same process for the next a account. This is a manual, time-consuming, costly and needless task.

So what is the problem?

Hierarchical Deterministic (HD) wallets have some interesting properties.  Firstly, you only need to backup the master key (e.g. as a 24 word mnewmonic).  From this all child keys can be derived again at any time.  Each coin/token has a well defined sub tree, as defined by the derivation path.  So Bitcoin has one standard path, and Ethereum another. Well, this is not strictly true, bitcoin has at least 2 standard paths, one for “legacy” and one for “segwit”, but that is another post.  Within a specific path, any number of child keys can be derived deterministically (i.e. if you start from the same master key, you will always get the same child keys).  Child keys have an index.  so 0 is the first, 1 is the second etc.  key 0 and key 1 are siblings.

The missing piece in HD wallets is knowing how many child keys were generated, i.e. what index you are on.  As you are only backing up the master key, not the master key + indexes of children for each path.

One solution is the “discover gap limit”.  This setting tells the SW wallet how many addresses for each parent to “load up” (i.e. show the balances for and allow transfers from).  For example a discovery gap limit of 20 means generate the first 20 addresses for the given parent (i.e. index 0 to index 19).  For each of these check the blockchain to see if there were any transactions. If any transactions are round, show the first 20 (with their addresses and balances), and look at the next 20 (index 20 to 39) and so on until no transactions are found for the given block.

The Trezor wallet has a discovery gap limit of 20, allowing 20 new (empty) addresses to be generated without the “loading hack“.

Ledger has a discovery gap limit of 1, alowing only a single new address tone generated (for a given token).

Electrum allows the discovery gap limit to be configured, so you could have 100 or 1000 new accounts to give to your customers in parallel.

The downside to this method is that if you are using a wallet with a high discovery gap limit, then you decide to move to a different wallet with a low discovery gap limit, AND restore your master seed to that new wallet, you wont “see” your accounts. They are there, no money is lost, but the wallet just wont let you view them or use them.

For this reason, all wallets should have a configurable discovery gap limit in the advanced settings.  This is, at most, a few days of development effort for the wallet providers. There is no excuse not to implement this, even as a paid for “pro” feature.

Trezor is far superior to Ledger in this regard (20 vs 1)

 

How to start/stop Postgres on Mac from oficial package install.

If you install Postgres via the oficial installer, i.e. from here  you will be surprised that the official documentation for starting and stopping the database does not work for Mac.

Googling will also not help. Most answers are for the brew install, which has a different layout.

Here is a step by step guide (there are hopefully better ways, but the is the only way found so far):

   1. find out where your database is stored.

If you can connect to the database, e.g. using the included pgadmin III or heidisql, then run this query:

select setting from pg_settings where name = ‘data_directory’

This should yield something like:

/Users/yourname/yourdb

    2.  change user & directory

As the required executables, namely Postgres and pg_admin, are not available to root or any other user you will need to switch to the Postgres user, then change directory:

$ sudo su – postgres
[enter your Mac password here]
$ cd bin

     3. run command:

$ ./pg_ctl -D /Users/yourname/yourdb stop

$ ./pg_ctl -D /Users/yourname/yourdb start -l /path/to/server.log &

What should path to log file be?  Very good question. looking at available posts and documentation which do not work because the path does not exist, e.g:

/usr/local/var/postgres/server.log
/Applications/xTuple/postgresql/data

Searching the entire system for .log files, it would appear that the default install is putting rotated logs in:

/Users/yourname/yourdb/pg_log

log files look like: postgresql-2018-10-08_211024.log

If anyone knows how to start Postgres with these rotated log files, let me know.

The next problem is how to disable and enable proges’s autostart on boot feature for Mac.