grails 3: custom validators and custom error messages

Adding custom validators to grails domain objects is easy. Consider the following domain class:

class Transaction {
    Account account
    BigDecimal debit = 0
    BigDecimal credit = 0

    static constraints = {
        debit nullable: false, min: 0.0, validator: {val, obj -> if ((val == 0.0 && obj.credit == 0.0) || (val > 0.0 && obj.credit > 0.0)) return "transaction.equal" else return true}
        credit nullable: false, min: 0.0
}

This shows how the valiator closure is checking that either credit is not zero or debit is not zero:

validator: {val, obj -> 
     if ((val == 0.0 && obj.credit == 0.0) 
         || (val > 0.0 && obj.credit > 0.0)) 
         return "transaction.equal" 
         else return true
}

The error message to be displayed to the user if the constraint fails will be looked up the following file:

grails-app/i18n/messages.properties

Edit this file and add your message property:

transaction.equal="Either debit or credit must be > zero, but not both"
Advertisements

Grails 3: using enums with Domain Classes

It is common to need somethingStatus or somethingType in your domain classes. e.g. Account.accountStatus might be “open”, “closed”, “suspended” etc.

There are two ways to implement this:

  1. using a separate domain class
  2. using a string or int enum.

If the status has other properties, then use a separate domain class. E.g.

Account.groovy:

class Account {
  User user
  Currency currency
  AccountStatus accountStatus
}

AccountStatus.groovy:

Class AccountStatus {
   String name
   boolean canLogin
}

However, in simple cases, you can use an enum:

Account.groovy:

class Account {
  User user
  Currency currency
  AccountStatus accountStatus
}

enum AccountStatus {
   OPEN,
   CLOSED,
   SUSPENDED
}

Here is an example of usage:

Bootstrap.groovy:


def account1 = new Account(user: user1, currency: gbp, accountStatus: AccountStatus.OPEN).save(failOnError: true)

Grails 3.3. How to fix rounding and truncation of BigDecimal in Fields plugin with scale of more than 3.

If you requite more than 3 decimal places, e.g. to support display crypto currencies (ETH has 18 dp, BTC has 8), and use any of the scaffolded views or fields plugins such as the excellent f:table, f:display etc, you will notice that values of say “1.123456” are shown as “1.12”. Worse still, if you want to update the object using scaffolded views or the fields, even if you don’t change the value, it will be overwritten in the database with the truncated value.

The first step to fixing this is to override the widget used to display either all BigDecimals, or just the specific controller field, or the specific domain object (its your choice).

The documentation to refer to is: https://grails-fields-plugin.github.io/grails-fields/latest/guide/index.html

Lets say this is your domain object:

package me
class Account {
    User user
    BigDecimal balance = 0
    Currency currency
    static constraints = {
        balance nullable: false, scale: 18, precision: 50
    }
}

to override the display of all BigDecimals, create the file:

grails-app/views/_fields/bigDecimal/_displayWidget.gsp

The file should contain something like this:

${value.stripTrailingZeros().toPlainString()} 

Now you should see “1.123456”.

Note:

  1. If you don’t stripTrailing zeros, you will see “1.123456000000000000” if you have scale set to 18 for example.
  2. You can also override the display for a specific field or specific controller, rather than all big decimals (see docs).

Trezor vs Ledger HW wallet for business

There are several comparisons of the trezor and Ledger HW and their corresponding SW wallets. These mainly focus on the end user usability and number of coins supported.

However, when it comes to business, who want to support many customers, and for developers who want to integrate the HW wallets into existing commerce or financial systems, the picture is very different.

The conclusion is that Trezor is far better suited to business than the Ledger, but neither are perfect.

Note, here we don’t make the distinction between Ledger Nano S and Ledger Blue, and likewise between Trezor one and Trezor T, because the later models are basically the same as the former just with larger screen and a few extras.

Architecture

Trezor has everything built into the firmware.  The 500 or so coins it supports are always there and always avialble.  No installation or configuration is required. When Trezor add more coin support, you just upgrade the firmware (which takes less than one minute) and you are done.

Out of the box, the ledger has literally nothing on it, it cant do anything.  You need to install apps using their SW wallet.  Each coin is an app. So there is an app for BTC, one for ETH etc.  You can only install a small number of apps at the same time (something like 4 on the nano and 18 on the blue depending on app size).  If you want to do transactions in more than the allowed number of apps, you have to uninstall and reinstall.

When you upgrade the Ledger, you have to remove all the apps, then reinstall them.

APIs

Trezor has a single unified API which is coin agnostic – e.g. you can generate addresses for any coin with one call, and you can sign transactions for any coin.  The coin (e.g. Bitcoin or Ethereum) is just a parameter.

There are several ways to access the Trezor API. there is a python based command line tool, which allows you to do pretty much anything with a bash script or command line (such as generating 1000 addresses for a given path).  There is also the bridge, which allows web apps to access the Hw through the browser.  This then requires no SW to be installed on the users machine to

Ledger doesn’t have an API as such.  In theory, each app can provide its own API. There is apparently a beta API for the BTC app.  But there is nothing to say that other apps (aka coins) will get APIs, or if they will be unified in any way.

Compatibility

Interestingly, you can restore your 24 word seed on either device.  So they are largely compatible in this regard.  The only issue is that the Ledger uses a different Path for Ethereum than the Trezor by default, so you will need to use a wallet which allows the path to be configured (such as Electrum).

Labels

Labelling accounts and addresses is important for avoiding mistakes and for reconciliation/accounting.  e.g. if you have 10 addresses, you might label them with the 10 customer names you have given them to. Or you might label them for different invoices.

Trezor has an interestingly solution for labels. They dont want to store them on the device, as they will be lost if you lose the Trezor and need to restore it from the Seed. So instead the store the labels in an encrypted file in dropbox. The nice part of this is that other users (in a company) can use their dropbox to securely share the labels between administrators.

Ledger supports labels for accounts, but doesn’t support addresses for accounts so these cannot be labeled.

Trezor can label individual addresses.

Discovery gap limit, accounting, and client funds segregation.

If you have say 20 customers, and you want to invoice each, you would want to give each customer their own account in the form of an address.  This allows proper client fund segregation, easier audit and accounting.  Even better, an account per invoice.  Additionally, you would want to label each account with its function.  At a glance, you can see the balance and transactions on any account.

This should be trivial to do with the HD wallet, as it supports unlimited addresses for a given account path. However, due to the limited SW implementations of the discovery gap, it is not.

Ledger only allow one address to be generated at a time, and only for one account.  It ignores one of the main features of the HD wallet – the ability to generate many addresses, with increasing indexes, for a given account.  It is difficult to compare the ledger wallet to any other wallet, as its feature set is so limited.

Ledger_Live

Above we see an “account” called trstbtc1.  It has one address, which corresponds to index zero. You cannot see this address.

 

You cannot add a second address until there is a transaction on the first.  If you have two customers, and want to give each a separate address for payment, you cant.  You would have to give the first customer an address, wait for them to pay, then the SW will allow you generate the second address.   This goes for their “accounts” also.  The work around I see clients doing every day is the “loading hack“. They generate an addresss, transfer 1 satoshi (or equivalent coin) in to it, wait till it is confirmed (1 hour in the case of BTC), then they can generate a second address.  Next they remove the 1 satoshi, so the customer who gets that address given to him doesn’t get worked that it has a balance (so is a “used” account), and do the same process for the next a account. This is a manual, time-consuming, costly and needless task.

So what is the problem?

Hierarchical Deterministic (HD) wallets have some interesting properties.  Firstly, you only need to backup the master key (e.g. as a 24 word mnewmonic).  From this all child keys can be derived again at any time.  Each coin/token has a well defined sub tree, as defined by the derivation path.  So Bitcoin has one standard path, and Ethereum another. Well, this is not strictly true, bitcoin has at least 2 standard paths, one for “legacy” and one for “segwit”, but that is another post.  Within a specific path, any number of child keys can be derived deterministically (i.e. if you start from the same master key, you will always get the same child keys).  Child keys have an index.  so 0 is the first, 1 is the second etc.  key 0 and key 1 are siblings.

The missing piece in HD wallets is knowing how many child keys were generated, i.e. what index you are on.  As you are only backing up the master key, not the master key + indexes of children for each path.

One solution is the “discover gap limit”.  This setting tells the SW wallet how many addresses for each parent to “load up” (i.e. show the balances for and allow transfers from).  For example a discovery gap limit of 20 means generate the first 20 addresses for the given parent (i.e. index 0 to index 19).  For each of these check the blockchain to see if there were any transactions. If any transactions are round, show the first 20 (with their addresses and balances), and look at the next 20 (index 20 to 39) and so on until no transactions are found for the given block.

The Trezor wallet has a discovery gap limit of 20, allowing 20 new (empty) addresses to be generated without the “loading hack“.

Ledger has a discovery gap limit of 1, alowing only a single new address tone generated (for a given token).

Electrum allows the discovery gap limit to be configured, so you could have 100 or 1000 new accounts to give to your customers in parallel.

The downside to this method is that if you are using a wallet with a high discovery gap limit, then you decide to move to a different wallet with a low discovery gap limit, AND restore your master seed to that new wallet, you wont “see” your accounts. They are there, no money is lost, but the wallet just wont let you view them or use them.

For this reason, all wallets should have a configurable discovery gap limit in the advanced settings.  This is, at most, a few days of development effort for the wallet providers. There is no excuse not to implement this, even as a paid for “pro” feature.

Trezor is far superior to Ledger in this regard (20 vs 1)

 

How to start/stop Postgres on Mac from oficial package install.

If you install Postgres via the oficial installer, i.e. from here  you will be surprised that the official documentation for starting and stopping the database does not work for Mac.

Googling will also not help. Most answers are for the brew install, which has a different layout.

Here is a step by step guide (there are hopefully better ways, but the is the only way found so far):

   1. find out where your database is stored.

If you can connect to the database, e.g. using the included pgadmin III or heidisql, then run this query:

select setting from pg_settings where name = ‘data_directory’

This should yield something like:

/Users/yourname/yourdb

    2.  change user & directory

As the required executables, namely Postgres and pg_admin, are not available to root or any other user you will need to switch to the Postgres user, then change directory:

$ sudo su – postgres
[enter your Mac password here]
$ cd bin

     3. run command:

$ ./pg_ctl -D /Users/yourname/yourdb stop

$ ./pg_ctl -D /Users/yourname/yourdb start -l /path/to/server.log &

What should path to log file be?  Very good question. looking at available posts and documentation which do not work because the path does not exist, e.g:

/usr/local/var/postgres/server.log
/Applications/xTuple/postgresql/data

Searching the entire system for .log files, it would appear that the default install is putting rotated logs in:

/Users/yourname/yourdb/pg_log

log files look like: postgresql-2018-10-08_211024.log

If anyone knows how to start Postgres with these rotated log files, let me know.

The next problem is how to disable and enable proges’s autostart on boot feature for Mac.

 

 

 

 

 

Cut and paste between macOS and linux guest with Virtualbox.

If you have installed a server version of Linux, without a GUI, but still want to be able to cut and paste text and commands, then follow these steps.

Note, the default terminal you get with virtualbox does not offer cut and paste, you will need to use ssh or ssh client such as putty which support cut and paste. To use ssh or ssh client, you need to be able to connect to your virtual machine on port 22.

How you do this will depend on which type of network adapters you have configured.  The default for Virtual box is NAT

  1. NAT.  Guest can see internet. Host cant see guest.  Guest guest a private IP.
  2. Host.  Guest cannot see internet. host can see guest on private network.
  3. Bridged. Guest can see internet as if it was a physical server connected to your router, and gets an IP on the same network as the host. Host can see guest.

For non NAT setups, skep the next step.

1. Setup port forwarding on VirtualBox.

  1. run your vm, login to the console and do “sudo ifconfig”.  Look for the ip address, it should be something like 10.0.2.15
  2. open the virtualbox control panel
  3. right click on your VM and select “settings”
  4. Click on “Network” icon
  5. this assumes you only have one adapter of type NAT.
  6. Click on Advanced
  7. Click on “port forwarding”
  8. Add a rule as follows:
    1. Protocol: TCP
    2. Host IP: 127.0.0.1
    3. Host Port: 2222
    4. Guest IP: 10.0.2.15
    5. Guest port: 22
  9. Hit OK/Apply

2. Install SSH

check if you have ssh installed:

$ sudo service ssh status

If not install it:

$ sudo apt-get install openssh-server

3. login with ssh

Now you have your port redirection setup, and ssh running on your guest, you can ssh in using a client which supports cut and paste, such as putty, kitty (windows), ssh and iterm (mac).

Cut and paste with the usual Command-C Command-V and selections with the mouse.

4. Installing Guest Additions.

As a bonus, you may want to install guest services. This apparently improves performance, and allows cut/paste in the GUI installs of linux (don’t forget to enable bidirectional cut and paste in Virtualbox settings also).

  1. In the virtualbox guest settings, click Devices | Insert Guest Additions CD Image.
  2. Log in to your guest server.
  3. sudo mount /dev/cdrom /media/cdrom
  4. cd /media/cdrom.
  5. sudo ./VBoxLinuxAdditions.run

5. Bonus: setting up shuttle to remember your ssh settings

If you, like me, have hundreds of ssh server settings, you probably can’t remember them all.   For windows users there is the excellent Putty, and even better kitty.  Mac users have a poor selection of SSH managers, but shuttle is the best I have found so far. There is no GUI to setup the servers, but there is a very nice dropdown menu of servers once you have added them to the shuttle config file.  here is a excerpt for mine:

{ “Virtualbox Servers”: [
{
“cmd”: “ssh -p 2222 youruser@localhost”,
“inTerminal”: “tab”,
“name”: “VM2”,
“theme”: “basic”,
“title”: “Virtualbox VM2 test skywire”
}
]},

 

 

Limitations of Bitrix24 as a project management tool

There are many tools out there, from Jira at one end of the complexity spectrum, to Asana and similar tools at the other.  Most are focused only on task management, and lack the other tools corporations need, including employee directories, wikis, document repositories, messaging, workflow, invoicing etc.

We are currently trialing Bitrix24. Its main focus is CRM, but it also has pretty decent project management features, including subtasks.  It is also far cheaper than the competition ($40/m for 25 users). Compare this with Jira + confluence + various needed plugins = > $40/m per user.

NOTE: we only use the “cloud” version.  If you install and maintain bitrix on your own servers, at significantly higher cost, more customisation is possible.

Rather than go into detail about what it can do, below are the limitations we faced:

  1. No custom fields for users.  E.g. we use telegram, wechat, discord and github across our geographically diverse teams. We want every user to put his telegram ID, wechat ID, github ID etc. in their profile, as maintaining these lists is difficult.  Unfortunately, Bitrix is limited to skype, twitter, facebook, linked in and xing only.  There is no option to add additional fields or social networks/tools.
    1. social
  2. Custom fields cannot be added to all tasks for all projects.  Bitrix tasks are missing some key fields such as priority (which could be a number from 1 to 5, or a string such as “high” “medium” or “low”).  It is easy to add a custom field such as priority to a single task, but this is of little use.  There is no concept of adding fields to tasks for all projects, or for specific projects.  There is an option “set field set to all users”.  This has the rather unfortunate side effect of:
    1. Erasing any custom field anyone else has created.
    2. Applying this field to every task in the system, including peoples private tasks, workflow tasks, workgroup tasks, i.e. tasks outside of the project management system.
    3. If the person who created the custom task fields leaves the company, noone can maintain the custom tasks. Another admin will need to create a similar set, and apply this to everyone.  We have not tested what happens to the existing fields in this case.
    4. Custom fields cannot be made mandatory.
    5. There is the concept of task templates, but again, if you add a custom field to a task template, you need to overwrite every other list of custom fields.
  3. Invoicing only works for physical products, not services.  You can only have products which are measured in “m”, “l” “g”, “kg” and “pcs”. E.g. you cant bill for a specific number of days or hours at a specified rate.  As most of my clients only bill for consultancy, this is a significant limitation.
    1. unit
  4. Tasks do not have levels of priority, only “high priority” or not.  At minimum we would need “critical”, “high”, “medium”, “low” and, ideally, “nice to have”.  There is no usable way to add this.  If adding custom fields via task templates was a tenable solution, the task search and sort system doesn’t support custom fields.  Without the ability to search/report/sort based on priority levels, even using custom fields is not a solution.
  5. Tasks cannot be reordered.  E.g. if you have “test task” then “develop task”, and test task is set as starting after develop task has finished, test task will still be shown on top of develop task.  When you have 30+ tasks, and no way to order them, you get spaghetti in your gantt charts and in your task lists.  The only work around is to make sure you create each task in exact order, which for most projects is impracticable.
  6. If you create a project with tasks, and the project goes into test, the test team will need to create tickets against that project for Bugs etc.  Unfortunately, you can only create tasks on a project. Bugs have must have (and searchable) fields such as version found in, bug type, severity level, component, how to reproduce etc.   When bugs are closed they need a status such as verified, wont fix, fixed, fixed in version X etc.  None of this is possible, unfortunately  Also, new bugs should not appear on the gantt chart as tasks until verified and assigned.