Bash: check if a zip or a rar file has password-protection

ZIP
crypted=$( 7z l -slt -- $file | grep -i -c "Encrypted = +" )
if [ "$crypted" -eq "1" ]; then
    protected=1
fi

RAR
unrar x -p- -y -o+ "$file" 1> /dev/null 2> /dev/null
if [ "$?" -eq "3" ] ; then
    unrar x -p$password -y -o+ "$file" 1> /dev/null 2> /dev/null
fi

Source: https://supportex.net/blog/2011/08/bash-check-zip-rar-file-password-protection/

Linux Tips

Lock screen: ‘xscreensaver-command’ -lock
Vietnamese keyboard: addon firefox / extension chromium

Good Proxy http://www.publicproxyservers.com/

VPN, OpenVPN http://www.vpnbook.com/freevpn

http://www.securitykiss.com/resources/download/mac/tunnelblick_tutorial/

 

Ubuntu screen resolution resizable: sudo apt-get install virtualbox-guest-dkms

multiple Python versions on Centos: http://toomuchdata.com/2014/02/16/how-to-install-python-on-centos/

 

virtualbox, network, nat, internet, DEFROUTE=no, don’t try to configure DEFROUTE=yes, centos6.0

Repositories Centos Redhat Fedora epel: http://www.rackspace.com/knowledge_center/article/install-epel-and-additional-repositories-on-centos-and-red-hat

Bash cut string substring

bashtrick

 

Create user, change password remotely

Changing Linux User’s Password in One Command Line

Reload static ip address (when the old IP adress still persists):
sudo ifdown <network interface> && sudo ip addr flush <network interface> && sudo ifup <network interface>

https://askubuntu.com/questions/829700/reload-static-ip-ubuntu-16

Koding : coding in the Cloud

https://koding.com [just calm down and move on, it’s gonna save your time later, trust me]

This one is new to me so I’m going to have a brief introduction here, hopefully it’s worth your time reading this :

1. The first thing is an active support from the Community & Administrators :
My question was answered in 5 minutes if not less :

x00_support

 

2. They give you the Terminal with “sudo”, where you can easily manage all of your apps :

x01_terminal

x01_terminal_

 

3. You’re not alone ! “Teamwork” is available for big projects with a lot of frameworks serve your needs

x02_teamwork4. Editor in town, code highlighter and syntax hinting :

x03_ACE_editor

5. DevTool for the ease of deployment :

x04_DevTool

6. Installing applications can’t never be easier, after few clicks, my app is up and running :

x05_Application

x06_install_django

x07_test_django

7. The Environment , where you not only have an overview of the topology but also deploy each of the components :

x08_environment

8. Checking the System Health lets you know what is going on under the hood :

x09_system_health

9. Pricing ? Well, it’s totally free for individual like me without so much demand of performance, hardware …etc. And I think the pricing is reasonable for those who pay for this service:

x10_pricing

 

This website gives you 3GB of space and you can earn up to 20GB if you introduce it to your friends (500MB each)

If you find this post useful, here is the registration URL:  https://koding.com

koding_referer

Muchas gracias !

Hadoop 2.2 Single Node Installation on CentOS 6.5

By far the best tutorial for you to get started with Hadoop installation.

Source : http://alanxelsys.com/2014/02/01/hadoop-2-2-single-node-installation-on-centos-6-5/

Introduction

This HOWTO covers Hadoop 2.2 installation with CentOS 6.5. My series of tutorials are meant just as that – tutorials. The intent is to allow the user to gain familiarity with the application and should not be construed as any type of best practices document to be used in a production environment and as such performance, reliability and security considerations are compromised. The tutorials are freely available and may be distributed with the proper acknowledgements. Actual screenshots of the commands are used to eliminate any possibility of typographical errors, in addition long sequences of text are placed in front of the screenshots to facilitate copy and paste. Command text is printed using Courier font. In general the document will only cover the bare minimum of how to get a single node cluster up and running with the emphasis on HOW rather than WHY. For more in depth information the reader should consult the many excellent publications on Hadoop such as Tom White’s – Hadoop: The Definitive Guide, 3rd edition and Eric Sammer’s – Hadoop Operations along with the Apache Hadoop website.

Please consult www.alan-johnson.net for an online version of this document.

Prerequisites

  • CentOS 6.5 installed

Machine configuration

In this HOWTO a physical machine was used; but for educational purposes Vmware Workstation or Virtualbox (https://www.virtualbox.org/) would work just as well. The screenshot below shows acceptable VM machine settings for VMWare.

Note an additional Network Adapter and physical drive have been added. Memory allocation is 2GB which is sufficient for the tutorial.

User configuration

If installing CentOS from scratch then select a user <hadoopuser> at installation time otherwise the user can be added by the command below. In addition create a group called <hadoopgroup>.

Note the initial configuration is done as user root.

=> passwd hadoopuser
to enable log-in for this one. 

Now make hadoopuser a member of hadoopgroup.

usermod –g hadoopgroup hadoopuser

Verify by issuing the id command.ss

id hadoopuser

The next step is to give hadoopuser access to sudo commands. Do this by executing thevisudo command and adding the highlighted line shown below.

Reboot and now log in as user hadoopuser.

Setting up ssh

Setup ssh for password-less authentication using keys.

ssh-keygen -t rsa -P ”

Next change file ownership and mode.

sudo chown hadoopuser ~/.ssh

sudo chmod 700 ~/.ssh

sudo chmod 600 ~/.ssh/id_rsa

Then append the public key to the file authorized_keys

sudo cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

Change permissions.

sudo chmod 600 ~/.ssh/authorized_keys

Edit /etc/ssh/sshd_config

Set PasswordAuthentication to no and allow empty passwords
as shown below in the extract of the file.

Verify that login can be accomplished without requiring a password.

Installing and configuring java

It is recommended to install the full openJDK package to take advantage of some of the java tools,

Installing openJDK

yum install java-1.7.0-openjdk*

After the installation verify the java version

java -version

The folder etc/alternatives contains a link to the java installation; perform a long listing of the file to show the link and use it as the location for JAVA_HOME.

Set the JAVA_HOME environmental variable by editing ~/.bashrc

Installing Hadoop

Downloading Hadoop

From the Hadoop releases page http://hadoop.apache.org/releases.html , download hadoop-2.2.0.tar.gz from one of the mirror sites.

Next untar the file

tar xzvf hadoop-2.2.0.tar.gz

Move the untarred folder

sudo mv hadoop-2.2.0 /usr/local/hadoop

Change the ownership with sudo chown -R hadoopuser:hadoopgroup /usr/local/hadoop

Next create namenode and datanode folders

mkdir -p ~/hadoopspace/hdfs/namenode

mkdir -p ~/hadoopspace/hdfs/datanode

Configuring Hadoop

Next edit ~/.bashrc to set up the environmental variables for Hadoop

# User specific aliases and functions

export HADOOP_INSTALL=/usr/local/hadoop
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
export PATH=$PATH:$HADOOP_INSTALL/sbin
export PATH=$PATH:$HADOOP_INSTALL/bin

Now apply the variables.

There are a number of xml files within the Hadoop folder that require editing which are:

  • mapred-site.xml
  • yarn-site.xml
  • core-site.xml
  • hdfs-site.xml
  • hadoop-env.sh

The files can be found in /usr/local/hadoop/etc/hadoop/. First copy themapred-site template file over and then edit it.

mapred-site.xml

Add the following text between the configuration tabs.

<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>

yarn-site.xml

Add the following text between the configuration tabs.

<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>

core-site.xml

Add the following text between the configuration tabs.
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>

hdfs-site.xml

Add the following text between the configuration tabs.

<property>
<name>dfs.replication</name>
<value>1</value>
</property>

<property>
<name>dfs.name.dir</name>
<value>file:///home/hadoopuser/hadoopspace/hdfs/namenode</value>
</property>

<property>
<name>dfs.data.dir</name>
<value>file:///home/hadoopuser/hadoopspace/hdfs/datanode</value>
</property>

Note other locations can be used in hdfs by separating values with a comma, e.g.

file:/home/hadoopuser/hadoopspace/hdfs/datanode, .disk2/Hadoop/datanode, . .

hadoop-env.sh

Add an entry for JAVA_HOME

export JAVA_HOME=/usr/lib/jvm/jre-1.7.0-openjdk.x86_64/

=> Actually you don’t need to configure JAVA_HOME here since you’ve already done that in ~/.bashrc

Next format the namenode.

. . .

Issue the following commands.

start-dfs.sh
start-yarn.sh

Issue the jps command and verify that the following jobs are running:

At this point Hadoop has been installed and configured

Testing the installation

A number of test files exist that can be used to benchmark Hadoop. Entering the command below without any arguments will list available tests.

The TestDFSIO test below can be used to measure read performance – initially create the files and then read:

hadoop jar /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 100

hadoop jar /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar TestDFSIO -read -nrFiles 10 -fileSize 100

The results are logged in TestDFSIO_results.log which will show throughput rates:

During the test run a message will be printed with a tracking url such as that shown below:

The link can be selected or the address can be pasted into a browser.

Another test is mrbench which is a map/reduce test.

hadoop jar /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar mrbench –maps 100

Finally the test below is used to calculate pi. The first parameter refers to the number of maps and the second is the number of samples for each map.

hadoop jar $HADOOP_INSTALL/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar pi 10 20

. . .

Note accuracy can be improved by increasing the value of the second parameter.

Working from the command line

Invoking a command without any or insufficient parameters will generally print out help data”

hdfs commands

hdfs dfsadmin –help

. . .

hadoop commands

hadoop version

Web Access

The location for checking the Namenode status is at localhost:50070/. This web page contains status information relating to the cluster.

There are also links for browsing the filesystem.

Logs can also be examined from the NameNode Logs link.

. . .

The secondary namenode can be accessed using port 50090

On line documentation

Comprehensive documentation can be found at the Apache website or locally using a browser by pointing it at $HADOOP_INSTALL/share/doc/Hadoop/index.html/

Feedback, corrections and suggestions are welcome, as are suggestions for further HOWTOs.

A little bit of Python, FTP and CentOS

Today I will approach the system using few lines of Python code :

– Python code : FTP client

– CentOS server :  FTP server

Let’s get started !

FTP uses 2 port : 21 and (sometimes) 20 if the server is in active mode – or a random port (we’ll see them in this posting)

First of all, you need to install vsftp & ftp on your Server, by using those commands

yum install -y vsftpd #this one is for ftp server
yum install -y ftp #this one is for ftp client
service vsftpd start

Check whether FTP service is listening on port 21

ftp1

Alright, that’s for the server-side, now at your client, open Python IDLE and run this

from ftplib import FTP
f = FTP('ip_address_my_ftp_server')
f.login('anonymous','anonymous') #I'm using anonymous login
#f.quit() #omit this line, I'll explain below

Let’s check the netstat again and you’ll see what’s happening here :

ftp2

 

There you go “113.170.87.128:24567” is my computer’s ip address and the port number which the ftp-server randomized it (passive mode).

And because f.quit() wasn’t executed, so that the connection is still open, that’s why you see the line “ESTABLISHED” above. If f.quit() executed – the connection will close before you can see anything happen.

Try as many time as you can, and take a look at this picture – I was using “f.quit()” to close a connection to the CentOS FTP server :

ftp3

 

Have fun, I will write more about this in the next few days, we’re just scratching the surface, but it’s fun, right?

Python setup tools easy_install Linux

Cai dat easy_install cho linux, unix :

I ran into a bit of trouble installing easy_install, after reading many useless articles, this is how I solved the problem,  actually it’s rather simple :

Type :

wget https://bitbucket.org/pypa/setuptools/downloads/ez_setup.py && python ez_setup.py

and you’re done .

 

Bonus : to install virtualenv, all you have to do is “easy_install virtualenv”

Install Backtrack 5 startx black screen hang up

Cài đặt Backtrack 5, bị treo sau khi “startx”

I encountered this problem whilst I was installing Backtrack 5 R3 using USB (multiboot)

Well, I did google it and really found some helps, but after all I had to fix it myself

If you have googled and try this but not sucess :

 

  1. Edit the file /etc/default/grub
  2. Find the line: GRUB_CMDLINE_LINUX_DEFAULT=”text splash nomodeset vga=791″
  3. Change it to: GRUB_CMDLINE_LINUX_DEFAULT=”quiet splash i915.modeset=1 vga=791″
  4. Save and close the file.
  5. Run update-grub to refresh grub.
  6. Reboot

I guess you got stuck in step 5, you could not update grub

bt boot

 

move your cursor to the top, BackTrack Text and press TAB to edit

A command line appears, all you have to do is replace

“text splash nomodeset vga=791”

by this one :

“quiet splash i915.modeset=1 vga=791”

ENTER, and you’re all set

just remember one thing, after installing is finish. edit /etc/default/grub and update-grub , then startx . 

By the way, the default password for root is toor

fedora 18 sound problem pulseaudio

after installing fedora 18 KDE and have it updated, I got this problem and solved it by workaround :

open command line interface and type :

pulseaudio -k #to kill the process

nohup pulseaudio -vv & > /dev/null # run it again and even you disconnect from the terminal, it still running

 

well, I have no idea why the reboot didn’t kill it (or may be it has been respawned after reboot), but anyway it worked and hope it helps :P

single click, double click to nagivate folder fedora 18

This solution works for fedora 18 (KDE), if you’re currently using other versions or distros, just give it a try anyway :

open command line interface :

vi ~/.kde/share/config/kdeglobals

add (below anything if exists) or modify this one :

[KDE]
SingleClick=false

log out, log in and you’re all set . It’s going to take you a double-click to navigate inside a folder or open an executable file .

 

Thanks to : Peterius

http://www.linuxquestions.org/questions/linux-newbie-8/single-click-double-click-icons-in-kde-561262/

Digital Ocean Review : best cheap cloud VPS

This one is by far the best VPS cloud service I’ve ever used

Website : WWW.DIGITALOCEAN.COM

Free registration, after that you will be able to see the Control Panel as a glance .

These are some hightlighted features I’ll take you through :

  1. Awesomely CHEAP : 0.007$ per one hour, 5$  for a month of use, with  a powerful server : RAM 512MB, CPU 2-3GHz, 20GB hard disk driver SSD, 1 Terabyte Bandwidth .do_1
  2. Five most popular distros : Arch, CentOS, Debian, Ubuntu và Fedora (x86 and x64)
  3. 3 Servers reside in New York, San Francisco and  Amsterdam .
  4. Downloading speed from  SpeedTest is about > 10 MB/s, it depends on internet traffic and will have a much better performance  (click to resize this picture below)do_5
  5. You pay for what you used, after 4 hours  3 cents has been deducted (just 0.03 $, so cool !)do_2
  6. Brilliant Support , takes them two hours or half of a day to reply your question ticketsdo_3
  7. Anytime backup whenever you want to, they also offer automatic backup every day for you, totally no hidden fee .
  8. Takes approximately half minute to initialize your VPS.  Create as many server as you want to (default is five, but you can increase this limitation for free) . They charge you for what you use . When you destroy your VPS, your money is safe-kept
  9. Google Authenticator for account 2-step verification,  KVM technology give you the option to control the VPS through line console over web interface, Commission for anyone you refer, API RESTful , automated  RSA  Key for your next VPS creation …do_4
  10. Paying through Credit card or Paypal, very fast and secure.And there will be much much more for you to find out at : Website : WWW.DIGITALOCEAN.COM