Heapdump of Apache Flink in yarn mode

It happens some times when the apache flink yarn instance got crashed because of memory crash. Like:

2017-02-08 16:04:53,272 INFO  org.apache.flink.yarn.YarnJobManager                          - Container container_e38_1486041380317_0381_01_000002 is completed with diagnostics: Container [pid=31161,containerID=container_e38_1486041380317_0381_01_000002] is running beyond physical memory limits. Current usage: 2.8 GB of 2 GB physical memory used; 4.3 GB of 4.2 GB virtual memory used. Killing container.
Dump of the process-tree for container_e38_1486041380317_0381_01_000002 :
    |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
        |- 31235 31161 31161 31161 (java) 1697 274 4501712896 729895 /usr/share/java/jdk1.7.0_67/bin/java -Xms1448m -Xmx1448m -XX:MaxDirectMemorySize=1448m -Dlog.file=/hadoop/yarn/log/application_1486041380317_0381/container_e38_1486041380317_0381_01_000002/taskmanager.log -Dlogback.configurationFile=file:logback.xml -Dlog4j.configuration=file:log4j.properties org.apache.flink.yarn.YarnTaskManagerRunner --configDir . 
    |- 31161 31159 31161 31161 (bash) 1 1 108654592 308 /bin/bash -c /usr/share/java/jdk1.7.0_67/bin/java -Xms1448m -Xmx1448m -XX:MaxDirectMemorySize=1448m  -Dlog.file=/hadoop/yarn/log/application_1486041380317_0381/container_e38_1486041380317_0381_01_000002/taskmanager.log -Dlogback.configurationFile=file:logback.xml -Dlog4j.configuration=file:log4j.properties org.apache.flink.yarn.YarnTaskManagerRunner --configDir . 1> /hadoop/yarn/log/application_1486041380317_0381/container_e38_1486041380317_0381_01_000002/taskmanager.out 2> /hadoop/yarn/log/application_1486041380317_0381/container_e38_1486041380317_0381_01_000002/taskmanager.err 

Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143​

 And its very much difficult to find anything from the log even after enabling the debugging logs.

So it can be solved from the heapdump if yarn jobs.

First enable the heapdump in flink-conf.yaml. Add the following line as:

env.java.opts: -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=<LOG_DIR>/flink_heap_dump.hprof

Once the apache flink yarn job got crashed collect the yarn job log that have the heapdump also as:

yarn logs -applicationId application_1391047358335_0041 > application_1391047358335_0041.log

Changing IMEI in emulator on Ubuntu

It happens many time that you need to change the imei within android emulator on Ubuntu. So the following steps will guide you to do the same. The files that requires modification for imei are as follows:

• emulator64-arm
• emulator64-mips
• emulator64-x86
• emulator-arm
• emulator-mips
• emulator-x86

And the location for those files is /sdk/tools within android sdk.

1. Download or install android sdk.
2. Install bless hex editor.
   
                    sudo apt-get install bless
   
   
3. Open one by one the above mentioned files in bless editor. For ex:
   
   
   

   
   
4. Enable search mode using Ctrl + F and change Hexadecimal to Text
5. Search content "CGSN". The content next to the CGSN is the IMEI in the left pane.
   
   
                            For ex: +CGSN.000000000000000.
   
   
6. Now change the 15 digit to the value as per your choice. For ex: 123456789123456
7. Save the file. And perform this operation with each file.
8. Create a device in emulator and check the imei.

That's it.

Good Luck !!!!!

Installing Bugzilla 4.0 on Ubuntu 12.04 and above with MySQL

 Installing Bugzilla 4.0 on Ubuntu 12.04 and above with MySQL

These are quick steps to install Bugzilla 4.0 on Ubuntu 12.04 and above, you can refer to The Official Bugzilla Docs for detailed steps if needed.

Install Perl

Check the version of Perl you have installed on your machine.

 

$ perl -v

Bugzilla runs with Perl 5.8.1 or higher. If you don't have Perl installed on your machine, install it.

$ sudo apt-get install perl

Install MySQL

Check the version of MySQL you have installed on your machine.

$ mysql --version

Bugzilla runs with MySQL 4.1.2 or higher. If you don't have MySQL installed on your machine, install it.

$ sudo apt-get install mysql-admin mysql-client mysql-server

Install Apache

Check if Apache web server is already installed.

$ apache2 -v

If you don't have Apache installed already, install it.

$ sudo apt-get install apache2

Download Bugzilla Tarball

Download the latest 4.0 tarball from the Bugzilla site into your download directory.

$ cd ~/Downloads
$ wget http://ftp.mozilla.org/pub/mozilla.org/webtools/bugzilla-4.0.tar.gz
$ sudo tar -xvf bugzilla-4.0.tar.gz


Move the Bugzilla directory to the default web server directory.

$ cd /var/www/
$ sudo mv /home/build03/Downloads/bugzilla-4.0 bugzilla




Change the permission and ownership of the bugzilla folder.

$ sudo chown -R www-data:www-data bugzilla
$ sudo chmod -R 755 bugzilla




Install Perl modules needed for Bugzilla

Check if you have all the Perl modules required for Bugzilla. It is a good idea to install all the required as well as the optional modules.

$ cd /var/www/bugzilla/
$ sudo ./checksetup.pl --check-modules


A lot of the Perl modules might be missing, first try to install all of them in one command

$ sudo /usr/bin/perl install-module.pl --all


Run the check setup script again to see which modules still need to be installed.

$ sudo ./checksetup.pl --check-modules


Install mod_perl & Apache2::SizeLimit modules if they did not get installed.

$ sudo apt-get install libapache2-mod-perl2 libapache2-mod-perl2-dev libapache2-mod-perl2-doc
$ sudo /usr/bin/perl install-module.pl Apache2::SizeLimit


Once all the Perl modules have been installed, run checksetup.pl again, this time without the --check-modules switch, this will create a file called "localconfig".

$ sudo ./checksetup.pl


$ sudo gedit localconfig
 
Change the following properties to
 
$db_pass = 'your password for bugs@localhost';


Create a User for Bugzilla for MySQL Access

Add the new user bugs@localhost with the same password that you entered in the localconfig file above.

Enable apache modules for perl and cgi:

$sudo a2enmod

and type perl and cgi one by one.


$ sudo useradd -d /home/bugs -m bugs
$ sudo passwd bugs


Create the bugs database

The bugs database will be used by the 4.0 installation of Bugzilla.

$ sudo mysql -u root -p
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| bugzilla3          |
| mysql              |
+--------------------+
3 rows in set (0.02 sec)

mysql> create database bugs;
Query OK, 1 row affected (0.00 sec)

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| bugs               |
| bugzilla3          |
| mysql              |
+--------------------+
4 rows in set (0.00 sec)

mysql> GRANT SELECT, INSERT, UPDATE, DELETE, INDEX, ALTER, CREATE, LOCK TABLES, CREATE TEMPORARY TABLES, DROP, REFERENCES ON bugs.* TO bugs@localhost IDENTIFIED BY 'your password for bugs@localhost';
Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

mysql> quit
Bye


Setup the Administrative User for Bugzilla

Rerun checksetup.pl. It reconfirms that all the modules are present, and notices the altered localconfig file, which it assumes you have edited to your satisfaction. It compiles the UI templates, connects to the database using the 'bugs' user you created and the password you defined, and creates the 'bugs' database and the tables therein.

If all goes well, it will ask you for an email for the Bugzilla Administrator account and the password for that account.

Configure Apache to server Bugzilla

Navigate to the Apache directory and edit the httpd.conf file

$ cd /etc/apache2/sites-available
$ sudo gedit default


Add the following to the default

<Directory "/var/www/bugzilla">
AddHandler cgi-script cgi
DirectoryIndex index.cgi
Options +Indexes +ExecCGI -MultiViews +SymLinksIfOwnerMatch +FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all
</Directory>


Restart Apache web server.

$ sudo service apache2 stop
$ sudo service apache2 start







Kindly let me any of your issues you are facing as per the above content.




Best of luck !!!!!

Switching between various versions of ruby on Ubuntu machine

Initially I feel very difficult to switch various ruby version installed on Ubuntu machine. But after collecting the following information it become easy. So the steps to change the ruby version is as follows:

1. Open the terminal window
2. Then type the following command:
   
   update-alternatives --config ruby
   
   
3. This will list all the installed versions of ruby. Like :
   
   There are 2 choices for the alternative ruby (providing /usr/bin/ruby).
     Selection    Path                Priority   Status
   ------------------------------------------------------------
   * 0            /usr/bin/ruby1.8     50        auto mode
     1            /usr/bin/ruby1.8     50        manual mode
     2            /usr/bin/ruby1.9.1   10        manual mode
   
   
4. And the console will wait for user input. Like:
   
   Press enter to keep the current choice[*], or type selection number:
   
   
5. Choose your desired ruby version and press enter.
   

So now you are ready to use various version of ruby without using rvm.

Enjoy!!

How to find table size in MySql using workbench?

If you want to calculate memeory usage of all yout tables in your database as we did in mysql admintrator tool then simply the following query:


SELECT table_name "Tables",
( data_length + index_length ) / 1024 /
1024 "Table Size in MB",
( data_free )/ 1024 / 1024 "Free Space in MB"
FROM information_schema.TABLES
where table_schema="YOUR SCHEMA NAME"

You need to change only the SCHEMA NAME only.

Try it and let me know your feedback.

Enjoy!!!

If you want to generate pdf from HTML on Ubuntu and try to search the tool for the same, you will find lots of tools for it. But wkhtmltopdf is the best one among them. 

So please follow these steps to install wkhtmltopdf along with qt4 that can convert rich javascript graphs into pdf.

Step 1: First install the dependencies

          sudo apt-get install openssl build-essential xorg libssl-dev libxrender-dev libqt4-dev qt4-dev-tools

Step 2: Install wkhtmltopdf

     sudo apt-get install wkhtmltopdf

Step 3: Test the installation

      wkhtmltopdf http://www.highcharts.com/demo test.pdf

If you are using highchart and facing issue related to segmentation fault then try after disabling animation as mentioned 

in the follwoing code:

<script type="text/javascript" charset="utf-8">
 
  var chart = new Highcharts.Chart({
      chart: { renderTo: 'container' },
 
      // These make it work nicely with wkhtmltopdf
      plotOptions: { series: { enableMouseTracking: false, shadow: false, animation: false } },
 
      xAxis: { categories: ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct'] },
      series: [{ data: [29.9, 71.5, 106.4, 129.2, 144.0, 176.0, 135.6, 148.5, 216.4, 194.1] }]
 
  });
 
</script>

Have fun with wkthltopdf !!!!