deploying a laravel app to a linux server

Prerequisites:

1. your mysql DB setup and your schema installed (or use artisan),  the mysql user you are going to put in your .env file must have be granted relevant permissions on the relevant database.
2. php >= 5.6 for apache
   1. This will depend on your hosting.  for my hosting, I had to add this line to the root .htaccess file: “AddType x-httpd-php56 .php”
3. php >= 5.6 for command line
   1. When you SSH to your server (e.g. using kitty or putty), see what version you have:

      $ bash-3.2$ php -v
      PHP 5.6.22 (cli) (built: Jun 9 2016 18:53:49)

   2. If the answer is not 5.6 or higher, contact your server hosting provider and ask them how to change the default.  For mine, I edited my root .bashrc and added the following line:

      alias php=/usr/bin/php-5.6

   3. Composer
      1. This one is a bit o a mystery.  There are conflicting instructions.  The official website suggest running the following at the command line:

      2. php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
         php -r "if (hash_file('SHA384', 'composer-setup.php') === '669656bab3166a7aff8a7506b8cb2d1c292f042046c5a994c43155c0be6190fa0355160742ab2e1c88d40d5be660b410') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"
         php composer-setup.php
         php -r "unlink('composer-setup.php');"

      3. the result is a composer.phar file.  However, the instructions to run it universally are like this: “# composer xxx” which will not work. I chose to add the following line to my .bashrc file:

         alias composer=/pathtomydir/composer.phar

      4. However, running “# composer xxx” results in the following warning:

         Warning: Composer should be invoked via the CLI version of PHP, not the cgi-fcgi SAPI

      5. To solve this, I found the cli version, and changed the alias to this:

         alias composer="/usr/bin/php-5.6-cli ~/composer.phar"

Deployment:

1. get the project onto a server.  E.g.
   1. ssh myserver.com
   2. mkdir versions
   3. cd versions
   4. git clone –depth 1 https://someone@app.deveo.com/SomeProject
2. Now you should have the following dirs:
   1. www (or public_html or similar)
   2. versions
3. copy the public dir to your www root (including dot files) = don’t use *):
   1. cd www
   2. cp -r ../versions/SomeProject/. .
4. Modify www/index.php (not Public/index.php)

Install dependencies.

1. cd yourprojectdir
2. run “composer install”
3. This will download a bunch of files.
4. Unfortunately for me, at the end I got this:

Generating autoload files
> Illuminate\Foundation\ComposerScripts::postInstall
> php artisan optimize
Content-type: text/html

<br />
<b>Parse error</b>: syntax error, unexpected 'class' (T_CLASS), expecting ident ifier (T_STRING) or variable (T_VARIABLE) or '{' or '$' in <b>/var/sites/d/dev.g amificationgiant.com/public_html/versions/platform-new/artisan</b> on line <b>30 </b><br />
Script php artisan optimize handling the post-install-cmd event returned with er ror code 255

The problem seems to be that composer is spawning a new shell which gets the default php which is 5.4, and there is nothing I can do about this.  Dead end for our hosting provider.

The solution is the following:

1. $ cd
2. $ mkdir bin
3. $ ln -s /usr/bin/php-5.6 ~/bin/php
4. edit .bashrc to have the following two lines:

alias composer="/usr/bin/php-5.6-cli ~/composer.phar"
export PATH=/var/sites/g/gamificationgiant.com/bin:$PATH

Now you can install dependencies:

1. $ composer install
2. $ composer dumpautoload -o
3. $ php artisan route:cache

The last item will give the following error:

Unable to prepare route [api/user] for serialization. Uses Closure.

setup environment

1. cd to the root of your app
2. $ cp .env.example .env
3. edit .env with your database details etc.
4. $ php artisan key:generate
   1. Do this only if you dont want to use an existing key in your .env
5. $ php artisan config:cache
   1. This has to be done after adding the key to .env.

Hopefully now your app will be working.

Trouble shooting

Cipher issues

If you see this in your “storage/logs/laravel.log” file:

'The only supported ciphers are AES-128-CBC and AES-256-CBC with the correct key lengths.' in /var/sites/d/dev.gamificationgiant.com/public_html/versions/platform-new/vendor/laravel/framework/src/Illuminate/Encryption/Encrypter.php:43
Stack trace:
#0 /var/sites/d/dev.xxxxx.com/public_html/versions/platform-new/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php(27): Illuminate\Encryption\Encrypter->__construct(NULL, 'AES-256-CBC')

Then check the following:

1. a valid cipher is set in .env APP_KEY
2. you have run “php artisan config:cache” AFTER setting the key.

 

 

 

Advertisements

Creating JSON APIs in GRAILS 3.3

Creating simple API’s which accept json or get parameters, and respond with json was very easy in grails 2.5.  Unfortunately, many things have subtly changed in grails 3 making porting an app rather difficult, especially as the 3.3 documentation still has 2.5 examples in many places.

Note:

1. The project you add the API to does not need to be created with the rest-api profile.
2. The rest-api profile does provide json views, which may or may not be easier (Ill try that another day)
3. The code below was written from memory, so there will be some typos etc.  Let me know if you find any and update accordingly.

Let us assume we want a simple POST API with loginUser (and other calls)

loginUser will send a JSON request body like this:

{"username":"bob", "password":"pass"}

and expect to get back the following format:

{"result":0, 
 "player":{"username":"bob", "firstname":"bob", "id":1234}, 
 "accounts":[{"balance":1.0, "currencyIso":"GBP", id:1234}]
}

an error would look something like this:

{"result":100, "message":"invalid parameters"}

Here we are using result as a return status/error code, and there are many error codes defined in the API.

Let us assume our domain objects are something like this:

User.groovy

package bla
class User {
  String username
  String password
  String firstname
}

Account.groovy

package bla
Class Account {
  User user
  BigDecimal balance
  String currencyIso
}

Step 1: register the URL

This used to be in Configuration/UrlMappings.groovy. Now it is defined in grails-app/controllers/web.admin/UrlMappings.groovy.

Add your URL thusly:

class UrlMappings {
  static mappings = {
  // stuff
  "/api/v1/user/login"(namespace: "v1", controller: "UserApi", action: "login")
  }
}

Step 2: create your controller (you can do this by simply creating the file or using the create-controller grails command).  Note, here I am putting two classes into one groovy file.  You can split them into two files if you prefer.

UserApiController.groovy

package bla
import grails.converters.JSON

/* The LoginCommand is just to make validating and parsing the JSON input easier
*/
class LoginCommand implements grails.validation.Validateable { 
  String username 
  String password 
  static constraints = { 
     // TODO: share the constraints with the domain object.
     username nullable:false, blank: false, size: 3..32 
     password nullable:false, blank: false, size: 3..32
  }
} // class LoginCommand

class UserApiController {
  static namespace = 'v1'

  def login(LoginCommand cmd) {
  try {
    if (cmd.hasErrors()) { 
        log.error("invalid parameters") 
        render(status: 400, contentType: 'application/json') {
          result 10 
          message "errors in parameters" 
        return
        } 
     } else { 
         // TODO: use a service which will update login failed counters etc
         User user = User.findByUsernameAndPassword(cmd.username, cmd.password)
         if (user == null) { 
            render(status: 404, contentType: 'application/json') { 
                result 100 
                message "could not find user with that username and password" 
            } 
         } else { 
            def userAccounts = Account.findAllByUser(user)
            render(status: 200, contentType: 'application/json') { 
               result 0 
               user {   
                  id user.id   
                  username user.username 
                  firstname user.firstname
               } 
            } 
       } // else all good 
    } // else params ok 
  } catch (Exception e) { 
    log.error("Caught Exception in UserApiController.login()", e) 
    render(status: 500, contentType: 'application/json') { 
        result 9999, 
        message e.toString() 
    } 
  }
}

To test this you can do something like:

http://localhost:8080/api/v1/User/login?username=asdf&password=asdf

Or if you want to use POST, I recommend using https://restlet.com/

The above code works but doesn’t yet return the account info.  Rendering domain objects by hand is easy but hard to maintain – you don’t want to have to duplicate this code in all your apis which return the same objects.  For this there are two options:

1. object marshallers
2. json views.

Object marshallers and easy and powerful and work in both grails 2.x and 3.x.  Be aware they are currently not working when you user render(){….} – this is broken. You will need to use JSONBuilder as above and use the respose text parameter.  An object marshaller for our account would be defined in the bootstrap.goovy file thusly:

DecimalFormat df = new DecimalFormat("##0.00"); 
JSON.registerObjectMarshaller(Account) { 
    return [balance: df.format(it.balance), currencyIso: it.currencyIso, id: it.id]
}

Then you can update the login method thusly:

render(status: 200, contentType: 'application/json') { 
  result 0 
  user {   id user.id   username user.username firstname user.firstname } 
  accounts userAccounts
}

 

You can restrict the API to post only using this at the top of the controller:

static allowedMethods = [login: "POST"]

Lastly, don’t forget to run this on SSL only in production to hide the password (not port 80 or 8080), and to encrypt the password in the database (e.g. using a beforeInsert interceptor)

 

How to use font-awesome in Grails 3.2

This was surprisingly difficult. There are several posts, but none were complete or working, so here is a complete version.

1. download font-awesome zip file from here http://fontawesome.io/
2. create a directory grails-app/assets/fonts*
3. unzip the contents of the zip into fonts.  so you should have something like:
   1. grails-app/fonts/css/font-awesome.min.css
   2. grails-app/fonts/fonts/FontAwesome.otf
   3. etc.
4. Edit application.css under grails-app/assets/stylesheets and add**:

    *= require css/font-awesome

5. edit build.gradle in the project root and add

assets {
 minifyJs = true
 minifyCss = true
 includes = ["fonts/*"]
}

Now restart your server, and try adding something like this in your gsp pages:

<i class="fa fa-camera-retro fa-4x"></i> fa-4x

And you should see a camera icon.

NOTE:

* personally, I would have put this in a directory called font-awesome rather than fonts, but every post on the web does it this way, and I am assuming they have reasons.

**surprisingly, this is intelligent enough to pull in the minified version, even though the name matches the non-minified file.  This method has the side effect of including the fonts on every page.  In theory, it would be possible to do it via direct include instead. e.g:

<link rel="stylesheet" href="/assets/fonts/css/font-awesome.min.css">

But this does not work. If anyone knows why, let  us know!

configuring grails 3.2 to work with mysql

Sadly, grails 3 uses yaml files for configuration, instead of the goovy files in previous versions.  In my opinion, this is a big step backwards for these reasons:

1. Yaml is yet another syntax to have to learn for no real benefit.  Groovy is the backbone of grails – every one who uses grails knows it (and most love it).
2. Existing 2.5 configurations are not compatible.
3. datasource.groovy was a place for all database stuff. Now database config is mixed in with every other config in the same file, which makes sharing configs trickier.
4. There are no non-trivial examples documented anywhere using the new format.
5. Yaml is very finicky, with invisible syntax errors. E.g. if you accidentally have a tab instead of a space, it will fail.  As we are a tab for indentation shop, this was especially painful.
6. Poor tools support for yaml compared with groovy, especially in Eclipse.  E.g. no automatic yaml formatting or converting tabs to spaces – you will need to edit every line when converting from the old to the new and replace tabs with spaces, or try find external tools.
7. The out of the box application.yaml fails validation with the online yaml validation tools.
8. For more complex structures, I find yaml syntax is much harder to manage than groovy DSL.  Especially restructuring and/or cut and pasting is a disaster.
9. Yaml is full of Gothas.  E.g. “file: c:\hello” is ok, but “file: c:” is not.
10. groovy dsl is more powerful

To make matters worse, the grails 3 documentation, to date, still only has the old 2.5 groovy database configuration information.

Here is how we did it:

1. edit build.gradle file in your main projects root.

dependencies {
 :
 runtime 'mysql:mysql-connector-java:5.1.33'

2. Edit your application.yml in grails-app/conf dir.  You will have to translate your old database.groovy into yaml.  We did this by trial and error.

environments:
  development:
     dataSource:
       dbCreate: update
       pooled: true
       url: jdbc:mysql://127.0.0.1:3306/xxx
       username: yyy
       password: zzz
       driverClassName: com.mysql.jdbc.Driver
       dialect: org.hibernate.dialect.MySQL5InnoDBDialect
       properties:
         maxActive: -1
         minEvictableIdleTimeMillis: 1800000
         timeBetweenEvictionRunsMillis: 1800000
         numTestsPerEvictionRun: 3
         testOnBorrow: true
         testWhileIdle: true
         testOnReturn: true
         validationQuery: "SELECT 1"

 

Grails 3, Eclipse and exploded plugins (aka sub projects)

Any medium to large project is likely to need more than one grails app.  E.g. a web admin app and an API app.  Each is a war file deployed to different servers with different firewall rules and security.   Both share the same services and domain objects.

With grails 2.5 life was easy, you just created an app and an exploded plug in with the common services and domain objects the same root dir, added one line to the app to be dependent on the plugin, and IDEs just picked it up and worked.  You could run the app from command line or within GGST or intellij and any changes to files in the core plugin and the app would both auto-load.

Getting this to work with Grails 3 has been tricky.  This post has a complicated solution which would not play nice with eclipse.  The official manual has some recommendations.

Firstly, setup eclipse like this.  Grails 3.2 and eclipse Neon 4.6

Secondly, create your project root directory, e.g. “myProject”

Thirdly, in the myProject directory, create two grails projects thusly:

$ grails create-app web-admin --profile=web
$ grails create-plugin core

Fourthly, edit the web-admin/build.gradle and add the following lines:

grails { exploded = true
        plugins { compile project(':core-services') 
        } 
}

This is more or less the same as for 2.5.

at this point you can go into web-admin and run “grails run-app” and it works.

Now lets go and create a domain object in the core project:

$ cd ../core-services
$ grails create-domain-class net.xxx.someClass

Filthy, add settings.gradle

$ cd ..
$ echo "include 'web-admin', 'core-services'" >> settings.gradle

Now win you do “grails run-app” in the web-admin project, you should see 1 domain under artifacts in the web page at http://localhost:8080

The final piece is to get this working in eclipse.

In eclipse, under File->Import->Gradle->Existing Gradle Project, chose the root directory (the one above the two projects).

You can even run the project by selecting web-admin->application->bootRun in the “Gradle Tasks” window, although I have not checked yet if the auto loading works this way.

Note, if you want to generate static copies of scaffolded controllers for domain objects, you will need to do it in the core plugin project, then move the resultant files to the web project.

e.g. if you want to run “grails generate-controller Book” you need to do this in the root folder of the core plugin project, then move BookController.groovy to the web project.

Visual Studio Code and node.js

In the old days, we did everything in Eclipse.  Java, HTML, XML, EJBs, CCS, javascript.  Then came Grails, and we used GGTS (and Intellij for the rich).  Now we are developing js and for node. The node plugin for eclipse is almost pointless, and offers no code completion for example.  For a long time we used notepad++ to do all our node development. It offers much better, but not perfect, code completion, and you can easily  start/stop node as a macro.  Some features were missing, such as the ability to create and name a file in the project directory (you had to either do this outside of notepad++, or create a new file, hit save as, then tediously navigate to your project directory again.

In search of a better IDE for node development, we are going to try “Visual Studio Code”.  We are not sure why MS is developing free tools for open source platforms, but cant complain.  We only hope they don’t drop developing it if there is no money in it.

Here is the list of steps to setup the environment we use:

1. Download visual studio code from here
2. Click on the extensions icon on the left, and add the following plugins:
   1. ESlint
   2. There are probably more, but we haven’t find anything else worth installing yet.
3. Get rid of the activity bar. Right click on it and select “Hide activity bar”.  It would be more useful if this bar had things like “Save”.
4. Install and configure eslint.  Sadly, the plugin doesn’t do this for you, and the instructions it gives are incorrect. It also failed to find my exising eslint installation and configuration files, I had to start again.
   1. open a cmd prompt and navigate to your node project directory.
   2. type: “npm install eslint –save-dev. This gives warnings “no repository field”, which can be ignored.
   3. type: “node_modules\.bin\eslint –init”
   4. Follow the instructions using the up/down arrows and enter key.
   5. You will also have to create a project package file if you have not already.

Visual Studio Code is not perfect.

1. There is no save icon, you have to tediously open the file menu if you are a mouse oriented user.
2. The autosave option is great, but, unlike intellij, there is no local history feature to get back that file you just accidently saved with the wrong edits.  This makes autosave very dangerous.  Sadly, the windows file history (which can save every 15 minutes) only works with an external drive. It would be fantastic if it could work with an internal drive instead, just to save local changes.
3. It doesn’t support SVN. I am sure that git is great for distributed projects with push/pull, but you cant beat SVN for simplicity and convenience of many components in one place.
4. It doesn’t really have the concept of a project.  However, you can open your node folder, which is similar.
5. Getting the eslint plugin is non trivial.  Eclipse + node plugin is much better in this regard.
6. Settings have to be handed edited. With notepad++, its all UI based, checkboxes, dropdowns etc.  The problem is if you want to change something, you have to read the manual. e.g. by default “hotExit” is enabled.  This is a disaster for me as I have two machines, one at home and one in the office.  All my files are on oneDrive, so I can simply carry on where I left off when I go to/from work. But with hotExit, it doesnt ask me to save any edited files when you exit VSC.  The setting is as follows “files.hotExit”: “onExit”. The problem is, there is no comment or hint at how to switch this off,  Even google cant help.

 

Conclusion.

Each edit has different strengths and weaknesses.  All are flawed.  The ultimate IDE would have the simplicity and power of notepad++ editor, the code completion VCS, and the auto-save with local history of intellij and everything else from eclipse (multi-project handling, UI, rich plugins, painless jslint integration etc)

Grails 3.2 and eclipse Neon 4.6

Intellij is the only IDE which officially supports Grails 3.x.  It costs an iphone a year just to edit your petstore project.

So what are the other options.  The excellent, and in my opinion, superior GGTS  is sadly no longer developed, and stopped support at Grails 2.5.

But you can edit and run grails 3.2 in eclipse Neon.  You don’t get the wizards, but this can be done on the command line.

Install

1. Install java 1.8 JDK (aka SE)
2. Install Grails
   1. download from here 
   2. unzip e.g. to c:\dev
   3. set GRAILS_HOME=C:\dev\grails-3.2.6
   4. add %GRAILS_HOME%\bin to PATH.
   5. see guide
3. Install Eclipse
   1. Download the latest eclipse from here. I chose the J2EE edition.
   2. Unzip it somewhere handy (e.g. c:\dev\eclipse)
   3. make a short cut of the c:\dev\eclipse\eclipse.exe and
   4. run it.
4. install Groovy in eclipse
   1. Help->Install new Software -> Add..
   2. Set location to: http://dist.springsource.org/snapshot/GRECLIPSE/e4.6/
   3. Note, if you have a version of eclipse newer than Neon, which is 4.6, update the URL above to match the eclipse version
   4. Select:
      1. Groovy Compiler 2.4 feature
      2. Groovy Eclipse feature
   5. hit Next, accept terms, wait till installed, restart eclipse, bla bla etc.
5. Install gradle support.  In theory, neon is supposed to have the eclipse buildship plugin installed, but mine didn’t.
   1. Help->Install new Software -> Add..
   2. Set location to: http://download.eclipse.org/buildship/updates/e46/releases/2.x
   3. select “buildship: Eclipse plug-ins for gradle”
   4. Hit Next, accept, wait, restart, bla bla

Now you should be ready to import a grails project.

Make one via the command line, e.g.

grails create-app bottom

Import he project into eclipse.

1. In eclipse: File->Import->Gradle->Existing Gradle Project
2. Select root of the grails app folder (bottom in this case)
3. Chose “Grade Wrapper”.  No idea what this is, but seems to work.  It creats a bunch of files in your project root dir.

 

Run the project.

At the bottom window in eclipse you should see “Gradle Tasks”

Under this, you should see your project, and several subdirectories.

One of them is “application”.  under this is “bootRun”.

Double click this. it should work (if nothing is running on port 8080)

See Ted Vinke’s article for more info.