Vejju: August 2018

Introduction

Vagrant provides easy to configure, reproducible, and portable work environments built on top of industry-standard technology and controlled by a single consistent workflow to help maximize the productivity and flexibility. A brief wave crest specific introductory presentation is also available.

Details

One can follow the below procedure to get and use a vagrant based box.

Prerequisites

Download latest vagrant VM image from: \\10.10.10.8\software\Vagrant\Platform2.0_1.5 to your local machine. One can refer to image version specific notes for more details.
Download prerequiste software to run vagrant. We need vagrant tool version 1.6.5 and Virtual Box version 4.3.18-96516-Win for this. These two softwares are also available at: \\10.10.10.8\software\Vagrant . After installing these two, one has to make sure these are present in PATH.

E:>echo %PATH%

C:\Program Files\Java\jdk1.6.0_23\bin;C:\Windows\system32;C:\Program Files\Code Collaborator Client;C:\Program Files\TortoiseSVN\bin;C:\Program Files\Oracle\VirtualBox;E:\HashiCorp\Vagrant\bin
Optional: Please install fastCopy tool to transfer the Vagrant Images quickly from network location to local working area. You can grab the installation file from: \\10.10.10.8\software\Vagrant\FastCopy211_x64.zip Usage and other details are at: http://ipmsg.org/tools/fastcopy.html.en

Creating Vagrant VM

One has to create a work directory for vagrant in his/her code checkout area, specifically at his Platform directory. For example,

If your code is present at E:\Code\PlatForm2.0, you directory structure would look like this after creating work area for vagrant.

One can follow the below steps to download/add, initialize, starting the box in windows command prompt. You can go to vagrant work area (e.g. E:\Code\Platform2.0\vagrantWorkArea) and execute the following. Let us assume you have downloaded vagrant image to your downloads directory.

$ vagrant box add {title} {DownlaodedImageLocation}

Example,E:\Code\Platform2.0\vagrantWorkArea> vagrant box add myBox C:\\Users\\foo\\Downloads\\Platform2.0_1.1

$ vagrant init {title}

Example:

E:\Code\Platform2.0\vagrantWorkArea> vagrant init myBox

This will create a file entitled "Vagrantfile", which is a config file for VM.

$ vagrant up

In our example,

E:\Code\Platform2.0\vagrantWorkArea> vagrant up

Note:

vagrant up may fail if you do not have localrepo directory in your home directory.

So create one like /home/wcuser/localrepo in the home directory

Example:

E:\home\wcuser\localrepo

One can use multiple VM boxes by creating different directories and repeat the steps mentioned above from each directory.

When creating a new vagrant box with latest image,

a) Please make sure you run the following commands at your windows command prompt from vagrant work area.

1. vagrant box remove NAMEOFVM E.g. vagrant box remove myVm

2. vagrant destroy

followed by

b) Deleting your vagrant work directory contents in source code area.
E.g. If your existing vagrant work area is at D:\Code\Platform2.0\VmWorkArea, please delete the contents (specifically a directory called .vagrant and a file called Vagrantfile)

c) Giving a new name to your vagrant VM. E.g.: If you had givem myVm as your VM name, please choose a new one like myNewVm

Accessing Guest OS (VM)

One can use putty to logon to this box with the following credentials.

Username: wcuser / root
Pass: wcuser /vagrant
Host: localhost
port: 2222

Please refrain from using root user on Vagrant VM box. wcuser was given sudo access for almost all activities.
Database Details

     Db Schema: localpaycental
     UserName :  wcuser/root
     Pass: wcuser@123/wcuser@123

ekmi Details

Instace Name: localekmi

If one is running multiple boxes on a given host machine, the port number for the second box will be 2200 or some thing like that. This will be shown when you run vagrant up command.

Building artifacts using gradle

Download gradle-1.10

Append gradle's /bin folder path to environment PATH variable

cd to the SVN check out folder (ex: D://Platform2.0)

Run the gradle build command: $ gradle -x test buildAll

Installing middlewares and gems

Connect to vagrant using putty as wcuser

Update the installation in vagrant box

Run the command: $ ./automation/opstools/copysoftware.sh

Install gems : $ ./automation/opstools/setupgems.sh

Installs binaries: $ cd /home/wcuser/automation/tools and $ /usr/local/bin/thor middlewaremgmt:software LocalVagrantplatform2.0

Note : In the new vagrant image, synced_folders are not included by default. If you are not able to see files under automation folder add these lines to _Vagrantfile files

config.vm.synced_folder "/home/wcuser/localrepo", "/home/wcuser/localrepo"
config.vm.synced_folder "./../devtools/automation", "/home/wcuser/automation"
config.vm.synced_folder "../orchestration/httpd", "/home/wcuser/httpd"

Restart vagrant using ( vagrant halt -f and vagrant up) commands

Deployment of artifacts

One has to pick any one of the below methods to update database and deploy the artifacts. Method A is a brief one where as Method B is in detail of each and every step.

Method A

$ dos2unix /home/wcuser/automation/tools/localEnvUp.sh
$ sh /home/wcuser/automation/tools/localEnvUp.sh
(Please enter 'wcuser' as password for wcuser during execution)

Method B

One time activity: $rm -f /opt/redis-2.6.7/bin/redis.pid (when you bring vagrant up for the first time)
To update DB, one can use below command

$ cd /home/wcuser/automation/tools

  $  dos2unix dbinitupdate.rb
  $  dos2unix profiles/*

$ New dbupdate (./dbinitupdate.rb LocalVagrantplatform20 5.2.1.58676 jdbc:mysql://localhost:3306/ false migrate all)
$ Old dbupdate ( ./executeFlyway.sh jdbc:mysql://localhost:3306/ false migrate LocalVagrantplatform20 (for migration) )

To deploy, one can use the following commands

$ cd /home/wcuser/automation/tools


  $  dos2unix profiles/*

  $  ruby platformDeployment.rb ymlFilename versionNumber ( ruby  platformDeployment.rb LocalVagrantplatform20 5.2.1.58676)

$ ./provision.sh LocalVagrantplatform20 local remote appinit (Enter 'wcuser' as password if asks)

This step (i.e. deployment of artifacts) needs to repeated every time you bring the VM up  or would like to deploy latest assets

Running Java Batch Job (Optional)

One can run java batch jobs using the following steps.

$ dos2unix /home/wcuser/automation/tools/localEnvBatchInit.sh
$ sh /home/wcuser/automation/tools/localEnvBatchInit.sh

It will take few minutes to bring the application up after deployment. Please wait for 10 minutes or so to access the following services.

Restart httpd

$ sudo cp httpd-static/* /etc/httpd/conf.d/

$ /etc/init.d/httpd restart

switch between branch and trunk

$ ln -s trunk/ httpd-static

$ unlink httpd-static

$ ln -s Branch/ httpd-static

Hot Deployment

The idea is to use the existing services from local vagrant VM with local tomcat (on windows). Please follow the below steps.

Step1: Stop mysql windows service which is running on your work station (laptop or desktop) and make it's starting mode is as manual.

Step2: Modify existing vagrant file in your vagrant work area to map VM's MySql port (3306) ,DE port (7186) and MasterDataService(MDS) port to the same port in host.

Add below two lines in vagrant file

config.vm.network :forwarded_port, guest: 3306, host: 3306

config.vm.network :forwarded_port, guest: 7186, host: 7186

config.vm.network :forwarded_port, guest: 8085, host: 8085

and reload the vagrant box.

Step3: Start local tomcat (on Windows) on port#9090.

After this, bring Vagrant VM up followed by the following command. (It selectively brings all services up except tomcat)

sh /home/wcuser/automation/tools/localMiddlewareUp.sh localWindowsIPAddress

Accessing Services

- Orchestration Api: http://localhost:8080/orchestration-api-1.0/services/
- Orchestration Web: http://localhost:8080/orchestration-web-1.0/wallet/
- Admin: https://localhost/adminv1/tpl/
- Static: https://localhost/
- Accessing DB from Host Machine:
  By default, vagrant based VM's database listening port is mapped to 3307. So, use the following to access it.
  Db Schema: localpaycental UserName : wcuser/root Pass: wcuser@123/wcuser@123
  Port : 3307
Bringing VM Down and Up
One can bring the Vagrant VM down & up with the following commands respectively. One has to issue these commands from windows command prompt at vagrant work area location.
$ vagrant halt -f #Shutdowns the VM
$ vagrant up # Starts the VM

In our example,

E:\Code\Platform2.0\vagrantWorkArea> vagrant halt -f
E:\Code\Platform2.0\vagrantWorkArea> vagrant up

VM Suspend & Resume

$ vagrant suspend #Saves VM state where it is.
$ vagrant resume #VM resumes from from where it was saved by suspend command..
A suspend effectively saves the exact point-in-time state of the machine, so that when you resume it later, it begins running immediately from that point, rather than doing a full boot.
Running Smoke Tests

A separate document is available for this.

Additional Information

Sharing data between Host and Guest

To transfer data between host and guest machines, one has to keep the files in the created work folder. These will appear automatically in /vagrant directory from guest OS.

For example, on Host

E:\Code\Platform2.0\vagrantWorkArea> dir

Directory of E:\workarea

07/03/2013 01:59 PM    <DIR>          .
07/03/2013 01:59 PM    <DIR>          ..
07/03/2013 01:07 PM    <DIR>          .vagrant
07/03/2013 01:59 PM    <DIR>          dataToBeCopied
07/03/2013 01:06 PM             4,454 Vagrantfile

The equivalent directory in guest looks like

[wcuser@localhost ~]$ ls -al /vagrant

total 13
drwxrwxrwx   1 vagrant vagrant 4096 Jul 3 08:29 .
dr-xr-xr-x. 23 root    root    4096 Jul 3 07:39 ..
drwxrwxrwx   1 vagrant vagrant    0 Jul 3 08:29 dataToBeCopied
drwxrwxrwx   1 vagrant vagrant    0 Jul 3 07:37 .vagrant
-rwxrwxrwx   1 vagrant vagrant 4454 Jul 3 07:36 Vagrantfile

Port Forwarding

Forwarded ports allow anyone to access a port on one's host machine and have all data forwarded to a port on the guest machine.

For example: If the guest machine is running a web server listening on port 80, one can make a forwarded port mapping to port 8080 (or anything) on one's host machine. One can then open browser to localhost:8080 and browse the website, while all actual network data is being sent to the guest.

By default, the following forwarded port mapping is configured in given vagrant box.

Guest	Host
8080	8080
8020	8020
9160	9160
61616	61616
6379	6379
1234	1234
443	443
3306	3307
8085	8085

Note: One can change the existing port mapping in the corresponding Vagrantfile of that box, which is present in the work area. One has to modify the mapping line which looks like the following and reload the vagrant box.

config.vm.network :forwarded_port, guest: 8080, host: 8080

To reload a vagrant box,one can issue the following from work-area directory.

shutdown the box using vagrant halt -f command and bring it up using the command: vagrant up

Updating Softwares in Vagrant Box

See Here

Disabling Password Prompt

See Here

Configure Mail Client (postfix)

See Here

Add JCE Support

Add Card Number generation key sets to Redis

Troubleshooting Issues

Troubleshooting FAQs

I have an issue with port#443 while mapping. How to solve this issue?

You can check which service is holding up that port by using netstat -tuplen command at the command prompt. You may close that to relieve that port. Perhaps, Skype might be the culprit.
- One has to un-check the alternative ports of incoming connections in Skype. As port# 443 will be used by Vagrant tool.
  Navigation Steps: Tools –> Options --> Advanced --> Connection
I have an issue with port#ABCD during/after mapping. How to solve this issue?
You can change the existing port mapping by changing it in the Vagrantfile which is present in your vagrant work area. One has to modify the mapping line which looks like the following and reload the vagrant box.
config.vm.network :forwarded_port, guest: 8080, host: 8080

To reload a vagrant box,one can issue the following from work-area directory.
- shutdown the box using vagrant halt -f command and
- bring it up using the command: vagrant up
  
  For some reason, if that port is not available on host machine, you can follow the below steps to debug the issue.
1. You can check if a given process listens on mentioned port with netstat command first. E.g. netstat -tuplen on guest machine. [i.e. VM]
2. Then, when you do vagrant up, vagrant tool shows list of ports it has mapped like the following.
```
[default] Forwarding ports... 
[default] -- 22 => 2222 (adapter 1) 
[default] -- 8000 => 8888 (adapter 1)
```
  This is the place to check whether your mapping got configured or not.
3. If that does not work, to get a verbose debug info from vagrant, please do the following at your windows command prompt. This will print verbose information at the command prompt.
  
  $ set VAGRANT_LOG=info
  $ vagrant up
4. If still problem persists, firewall might be the culprit. Although firewall is disabled on vagrant images, if you enable it for some reason, you have to add an exception to this port in firewall as shown below.
  As root, do
  
  $ /sbin/iptables -I INPUT -p tcp --dport 4444 --syn -j ACCEPT $ /sbin/service iptables save $ /sbin/service iptables restart
I want to turn off the default port mapping for a port (say port#8080), which bundled with vagrant image. How can I do it?

You can do that by changing the vagrant configuration file which bundled with the image. Go to C:\Users\foo\.vagrant.d\boxes\image\virtualbox\include\_Vagrantfile. Replace foo with your userid. Then, you can change the existing port mapping in that file and reload the vagrant box.

config.vm.network :forwarded_port, guest: 8080, host: 8080

To reload a vagrant box,one can issue the following from work-area directory.
- shutdown the box using vagrant halt -f command and
- bring it up using the command: vagrant up
How can I cut down the size of my vagrant box?
Please inspect the culprit using the following command and clean the unwanted directories appropriately.
sudo du -sh /* | sort -nr
If that does not solve the issue, you can run the the following command as wcuser.
sudo dd if=/dev/zero of=/tmp/ZEROS bs=1M ; sudo rm -f /tmp/ZEROS
I can not access vagrant database instance from work station?
By default, vagrant based VM's database listening port is mapped to 3307. So, use the following to access it.
Db Schema: localpaycentral UserName : wcuser/root Pass: wcuser@123/wcuser@123
Port : 3307
I can not create a new database schema as wcuser in VM. It throw an error like 'failed: " " command on localhost'. How to solve that?

One can grant root like access to normal user on mysql using below steps.
$ mysql -uroot -pwcuser@123
mysql> GRANT ALL PRIVILEGES ON *.* TO 'wcuser'@'localhost' IDENTIFIED BY 'wcuser@123' WITH GRANT OPTION;
mysql> flush privileges;
mysql> quit;
Activiti Db is not getting created when I bring local environment up. How to correct it?
Since activiti database instance creation is one time deployment activity, by default the creation option is disabled. So, one has to activate this option before bringing local environment up. So, please follow the below steps.

Step1: Open this file:

/home/wcuser/automation/tools/profiles/deployinstanceLocalVagrantplatform20.yml

Step2: Go to dbservices section and change active filed from false to true.

Before

dbservices:
- name: activiti
    version: 5.12.1
    active: false
    ip: localhost

After

dbservices:
- name: activiti
    version: 5.12.1
    active: true
    ip: localhost

Step3: After saving that file, go ahead with local environment up command. (i.e., sh /home/wcuser/automation/tools/localEnvUp.sh)
Step4:

Once it is done, please change active filed value to false from true.

There is a run time error "MySQLSyntaxErrorException: Access denied for user 'wcuser'@'localhost' to database 'activitidb'" while the application is being up. How to solve that?

One can grant root like access to normal user on mysql using below steps.
$ mysql -uroot -pwcuser@123
mysql> GRANT ALL PRIVILEGES ON *.* TO 'wcuser'@'localhost' IDENTIFIED BY 'wcuser@123' WITH GRANT OPTION;
mysql> flush privileges;
mysql> quit;
This has to be followed by bringing environment up again (i.e. using sh /home/wcuser/automation/tools/localEnvUp.sh)
When I run localEnvUp.sh command, I ran in to an error, which looks like "/usr/local/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:55:in `require': cannot load such file -- mysql (LoadError)". How can I stop that error?

Please run the following command at your VM's command prompt as wcuser.
$ sudo yum -y install mysql55-devel ; sudo gem install mysql

(or)

$ yum install mysql-devel gcc make; gem install mysql -- --with-mysql-config=/usr/bin/mysql_config

Now, you can bring the environment up again (i.e. using sh /home/wcuser/automation/tools/localEnvUp.sh)
Do I need to run appinit command every time to re-deploy latest artifacts?
Yes
Redis service is not running. Also it is not coming up when I run appinit command as well? What should i do?
Due to improper shutdown of vm instance, redis pid file will be present. During the next start up call of redis, it creates problems. So, please delete it using $rm -f /opt/redis-2.6.7/bin/redis.pid
Now, you can go ahead with sh /home/wcuser/automation/tools/localEnvUp.sh command to bring all services up.
How can I install percona tool in my vagrant image?

Please run the following command at your UNIX command prompt by logging in as wcuser in to your vagrant VM box.

$dos2unix /home/wcuser/automation/tools/installPercona.sh; sh /home/wcuser/automation/tools/installPercona.sh;
vagrant up hangs at "Waiting for VM to boot. This can take a few minutes". How to get rid of this?

Please enable the Virtualization (VT/VT-d) option in your motherboard's BIOS. It is present in the Security -> Virtualization menu. You may have to update your BIOS software by contacting IT if that option is not present.
If that BIOS option does not work for some reason, a way to find the actual culprit is to print vagrant's debug information using the following command and act accordingly.
$ set VAGRANT_LOG=info
$ vagrant upThis will print a verbose information at the command prompt. One can inspect it and get to know the root cause.
There is permissions issue while dealing with shared folders between Host and Guest machine for wcuser account. Is there something I can do about?

Yes, while sharing the fodlers itself, you can set user specific permissions explicitly on that shared folder as well in Vagrantfile.

For example, if you have some issue automation and localrepo directories, please add the following two lines below the line # config.vm.synced_folder "../data", "/vagrant_data"
in Vagrantfile which is present in your vagrant work area and reload the vagrant box.
config.vm.synced_folder "/home/wcuser/localrepo", "/home/wcuser/localrepo", :owner=> 'wcuser', :group=>'wcuser'
config.vm.synced_folder "./../devtools/automation", "/home/wcuser/automation", :owner=> 'wcuser', :group=>'wcuser'

To reaload the VM, please do the following at your windows command prompt from vagrant work area.

- shutdown the box using vagrant halt -f command and
- bring it up using the command: vagrant up
  
  More verbose explanation on this at: http://stackoverflow.com/questions/13169154/cannot-change-permissions-of-folders-within-vagrant-home-folder
sh localEnvUp.sh throws an error related to https-static-1.0.tgz. The error looks like "/usr/local/lib/ruby/1.9.1/fileutils.rb:1515:in `stat': No such file or directory - /home/wcuser/localrepo/httpd-static-1.0.tgz (Errno: ENOENT)". What went wrong here?

When you do a gradle cleanrepo, gradle removes all the artifacts. When you do gradle build next time, it will not build httpd-static unless you make some change in that. Automation scripts reports missing of that component here. So, if you see this error after gradle cleanrepo, please build this component explicitly, using gradle P --rerun-tasks command. This will bring httpd-static-1.0.tgz back in local repo. It should solve this error.
Default VM is consuming 2G of total RAM. Is is possible to restrict my VM to use 1G instead?
Yes, it is possible. In your vagrant work area, you should be able to find VM's config file called Vagrantfile. You can change the existing memory value mapping by uncommenting three lines. Check the below table.This step has to be followed by reloading of the vagrant box.

Before the change

    # config.vm.provider :virtualbox do |vb|
#   # Don't boot with headless mode
#   vb.gui = true
#
#   # Use VBoxManage to customize the VM. For example to change memory:
# vb.customize ["modifyvm", :id, "--memory", "1024"]
# end
#

After the change

   config.vm.provider :virtualbox do |vb|
#   # Don't boot with headless mode
#   vb.gui = true
#
#   # Use VBoxManage to customize the VM. For example to change memory:
     vb.customize ["modifyvm", :id, "--memory", "1024"]
   end
#

To reload a vagrant box,one can issue the following from work-area directory.

shutdown the box using vagrant halt -f command and
bring it up using the command: vagrant up

References

Connect to vagrant using putty as wcuser

Update the installation in vagrant box

Run the command: ./automation/opstools/copysoftware.sh
Install gems : ./automation/opstools/setupgems.sh
Installs binaries: cd /home/wcuser/automation/tools and /usr/local/bin/thor middlewaremgmt:software LocalVagrantplatform2.0

Monday, August 6, 2018

Vagrant

Introduction

Details

Prerequisites

Creating Vagrant VM

Accessing Guest OS (VM)

Building artifacts using gradle

Installing middlewares and gems

Deployment of artifacts

Restart httpd

Hot Deployment

Accessing Services

Bringing VM Down and Up

Running Smoke Tests

Additional Information

Sharing data between Host and Guest

Port Forwarding

Updating Softwares in Vagrant Box

Disabling Password Prompt

Configure Mail Client (postfix)

Add JCE Support

Add Card Number generation key sets to Redis

Troubleshooting Issues

Troubleshooting FAQs

I have an issue with port#443 while mapping. How to solve this issue?

I have an issue with port#ABCD during/after mapping. How to solve this issue?

I want to turn off the default port mapping for a port (say port#8080), which bundled with vagrant image. How can I do it?

How can I cut down the size of my vagrant box?

I can not access vagrant database instance from work station?

I can not create a new database schema as wcuser in VM. It throw an error like 'failed: " " command on localhost'. How to solve that?

Activiti Db is not getting created when I bring local environment up. How to correct it?

There is a run time error "MySQLSyntaxErrorException: Access denied for user 'wcuser'@'localhost' to database 'activitidb'" while the application is being up. How to solve that?

When I run localEnvUp.sh command, I ran in to an error, which looks like "/usr/local/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:55:in `require': cannot load such file -- mysql (LoadError)". How can I stop that error?

Do I need to run appinit command every time to re-deploy latest artifacts?

Redis service is not running. Also it is not coming up when I run appinit command as well? What should i do?

How can I install percona tool in my vagrant image?

vagrant up hangs at "Waiting for VM to boot. This can take a few minutes". How to get rid of this?

There is permissions issue while dealing with shared folders between Host and Guest machine for wcuser account. Is there something I can do about?

sh localEnvUp.sh throws an error related to https-static-1.0.tgz. The error looks like "/usr/local/lib/ruby/1.9.1/fileutils.rb:1515:in `stat': No such file or directory - /home/wcuser/localrepo/httpd-static-1.0.tgz (Errno: ENOENT)". What went wrong here?

Default VM is consuming 2G of total RAM. Is is possible to restrict my VM to use 1G instead?

References

Security Certificates

I can not create a new database schema as wcuser in VM. It throw an error like `'failed: " " command on localhost'`. How to solve that?

`Activiti` Db is not getting created when I bring local environment up. How to correct it?

There is a run time error "`MySQLSyntaxErrorException: Access denied for user 'wcuser'@'localhost' to database 'activitidb`'" while the application is being up. How to solve that?

Do I need to run `appinit` command every time to re-deploy latest artifacts?

Redis service is not running. Also it is not coming up when I run `appinit` command as well? What should i do?

How can I install `percona` tool in my vagrant image?

`vagrant up` hangs at "`Waiting for VM to boot. This can take a few minutes`". How to get rid of this?

`sh localEnvUp.sh` throws an error related to `https-static-1.0.tgz.` The error looks like "/usr/local/lib/ruby/1.9.1/fileutils.rb:1515:in `stat': No such file or directory - /home/wcuser/localrepo/httpd-static-1.0.tgz (Errno: ENOENT)". What went wrong here?