Friday, November 18, 2016

What is the use of enumerate function in python

enumerate() is one of the built-in Python functions. It returns an enumerate object. In our case that object is a list of tuples (immutable lists), each containing a pair of count/index and value. 

>>> choices = ['pizza', 'pasta', 'salad', 'nachos']
>>> list(enumerate(choices))
=> [(0, 'pizza'), (1, 'pasta'), (2, 'salad'), (3, 'nachos')]
So, in the for index, item in enumerate(choices): expressionindex, item is the pair of count, value of each tuple: (0, 'pizza'), (1, 'pasta'), ...
We may easily change the start count/index with help ofenumerate(sequence, start=0)
for index, item in enumerate(choices, start = 1):
    print index, item
or simply with a number as the second parameter
for index, item in enumerate(choices, 1):
    print index, item
Try the following code:

>>>with open ('/etc/passwd') as f1:
...    ab = f1.readlines()
...     for i,v in enumerate(ab, 1):        
...      print i,v

Thursday, November 17, 2016

File handling in Python Cheatsheet: All modes

  • r

    Opens a file for reading only. The file pointer is placed at the beginning of the file. This is the default mode.
  • rb

    Opens a file for reading only in binary format. The file pointer is placed at the beginning of the file. This is the default mode.
  • r+

    Opens a file for both reading and writing. The file pointer will be at the beginning of the file. 
    (cannot truncate a file)
  • rb+

    Opens a file for both reading and writing in binary format. The file pointer will be at the beginning of the file.
  • w

    Opens a file for writing only. Overwrites the file if the file exists. If the file does not exist, creates a new file for writing.
  • wb

    Opens a file for writing only in binary format. Overwrites the file if the file exists. If the file does not exist, creates a new file for writing.
  • w+

    Opens a file for both writing and reading. Overwrites the existing file if the file exists. If the file does not exist, creates a new file for reading and writing.
    (can truncate a file)
  • wb+

    Opens a file for both writing and reading in binary format. Overwrites the existing file if the file exists. If the file does not exist, creates a new file for reading and writing.
  • a

    Opens a file for appending. The file pointer is at the end of the file if the file exists. That is, the file is in the append mode. If the file does not exist, it creates a new file for writing.
  • ab 

    Opens a file for appending in binary format. The file pointer is at the end of the file if the file exists. That is, the file is in the append mode. If the file does not exist, it creates a new file for writing.
  • a+

    Opens a file for both appending and reading. The file pointer is at the end of the file if the file exists. The file opens in the append mode. If the file does not exist, it creates a new file for reading and writing.
  • ab+

    Opens a file for both appending and reading in binary format. The file pointer is at the end of the file if the file exists. The file opens in the append mode. If the file does not exist, it creates a new file for reading and writing.


Important


Always make sure you explicitly close each open file, once its job is done and you have no reason to keep it open. Because - There is an upper limit to the number of files a program can open. If you exceed that limit, there is no reliable way of recovery, so the program could crash. - Each open file consumes some main-memory for the data-structures associated with it, like file descriptor/handle or file locks etc. So you could essentially end-up wasting lots of memory if you have more files open that are not useful or usable. - Open files always stand a chance of corruption and data loss.




Tuesday, November 15, 2016

Unable to connect to RHN network - ping works but nslookup fails

You should have proper entries in /etc/hosts and /etc/nsswitch.conf

Usually, you'll want its hosts line to look like(in /etc/nsswitch.conf):

hosts:      files dns
Antoher file to check is /etc/resolv.conf
check the permissions of all these files.

Read: http://superuser.com/questions/704785/ping-cant-resolve-hostname-but-nslookup-can

Monday, November 7, 2016

Installing Python modules using pip

pip is a package management system used to install and manage software packages written in Python. Many packages can be found in the Python Package Index (PyPI). Python 2.7.9 and later (on the python2 series), and Python 3.4 and later include pip (pip3 forPython 3) by default.

Install steps: https://pip.pypa.io/en/stable/installing/

Wget https://bootstrap.pypa.io/get-pip.py

Then run the following:

python get-pip.py

Install paramiko using pip


For reference, paramiko which we install has a hard dependency on cryptography which states:
For Debian and Ubuntu, the following command will ensure that the required dependencies are installed:
$ sudo apt-get install build-essential libssl-dev libffi-dev python-dev
For Fedora and RHEL-derivatives, the following command will ensure that the required dependencies are installed:
$ sudo yum install gcc libffi-devel python-devel openssl-devel
#pip install paramiko


Friday, October 21, 2016

Reading and Writing Files in Python

Overview

In Python, you don't need to import any library to read and write files.

The first step is to get a file object.

The way to do this is to use the open function.

File Types

A file is usually categorized as either text or binary.

A text file is often structured as a sequence of lines and a line is a sequence
of characters.

The line is terminated by a EOL (End Of Line) character. 

The most common line terminator is the 
 , or the newline character. 

The backslash character indicates that the next character will be treated as a
newline. 

A binary file is basically any file that is not a text file. Binary files can
only be processed by application that know about the file's structure.

Open ( )

To open a file for writing use the built-i open() function. open() returns a
file object, and is most commonly used with two arguments.

The syntax is:
file_object = open(filename, mode) where file_object is the variable to put the
file object.

The second argument describes the way in which the file will be used.

Mode

The mode argument is optional; 'r' will be assumed if it’s omitted.

The modes can be:

'r' when the file will only be read

'w' for only writing (an existing file with the same name will be erased)

'a' opens the file for appending; any data written to the file is automatically
added to the end. 

'r+' opens the file for both reading and writing.
>>> f = open('workfile', 'w')
>>> print f
Next the file objects functions can be called. The two most common functions are
read and write.

Create a text file

Let's first create a new text file. You can name it anything you like,
in this example we will name it "newfile.txt".
file = open("newfile.txt", "w")

file.write("hello world in the new file
")

file.write("and another line
")

file.close()
If we now look in the newfile.txt, we can see the text that we wrote:
$ cat newfile.txt 
hello world in the new file
and another line

How to read a text file

To read a file, we can use different methods.

file.read( )

If you want to return a string containing all characters in the file, you can
use file.read().
file = open('newfile.txt', 'r')

print file.read()
Output:

hello world in the new file
and another line
We can also specify how many characters the string should return, by using
file.read(n), where "n" determines number of characters.

This reads the first 5 characters of data and returns it as a string.
file = open('newfile.txt', 'r')
print file.read(5)
Output:

hello

file.readline( )

The readline() function will read from a file line by line (rather than pulling
the entire file in at once).

Use readline() when you want to get the first line of the file, subsequent calls
to readline() will return successive lines.

Basically, it will read a single line from the file and return a string
containing characters up to 
.
file = open('newfile.txt', 'r')

print file.readline():
Output:

hello world in the new file

file.readlines( )

readlines() returns the complete ?le as a list of strings each separated by 
file = open('newfile.txt', 'r')

print file.readlines()
Output:

['hello world in the new file
', 'and another line
']

Looping over a file object

For reading lines from a file, you can loop over the file object. 

This is memory efficient, fast, and leads to simple code.
file = open('newfile.txt', 'r')

for line in file:
    print line,
Output:

hello world in the new file
and another line

file.write( )

The write method takes one parameter, which is the string to be written. 

To start a new line after writing the data, add a 
 character to the end.
file = open("newfile.txt", "w")

file.write("This is a test
")

file.write("And here is another line
")

file.close()

Close ( )

When you’re done with a file, call f.close() to close it and free up any system
resources taken up by the open file. 

After calling f.close(), attempts to use the file object will automatically fail.

File Handling Usages

Let's show some example on how to use the different file methods
To open a text file, use:
fh = open("hello.txt", "r")
To read a text file, use:
fh = open("hello.txt","r")
print fh.read()
To read one line at a time, use:
fh = open("hello".txt", "r")
print fh.readline()
To read a list of lines use:
fh = open("hello.txt.", "r")
print fh.readlines()
To write to a file, use:
fh = open("hello.txt","w")
fh.write("Hello World")
fh.close()
To write to a file, use:
fh = open("hello.txt", "w")
lines_of_text = ["a line of text", "another line of text", "a third line"]
fh.writelines(lines_of_text)
fh.close()
To append to file, use:
fh = open("Hello.txt", "a")
fh.write("Hello World again")
fh.close
To close a file, use
fh = open("hello.txt", "r")
print fh.read()
fh.close()

With Statement

Another way of working with file objects is the With statement.

It is good practice to use this statement. 

With the "With" statement, you get better syntax and exceptions handling. 
In addition, it will automatically close the file. The with statement provides a
way for ensuring that a clean-up is always used.
Opening a file using with is as simple as:
with open(filename) as file:
Let's take a look at some examples
with open("newtext.txt") as file: # Use file to refer to the file object
    data = file.read()
    do something with data
You can of course also loop over the file object:
with open("newfile.txt") as f:
    for line in f:
        print line,
Notice, that we didn't have to write "file.close()". That will automatically be
called.

With Examples

Let's show some examples on how we can use this in our every day programming.

Write to a file using With

Write to a file using the With statement
with open("hello.txt", "w") as f:
 f.write("Hello World")

Read a file line by line into an list

This will read the file hello.txt and save the content into "data".
with open(hello.txt) as f:
    data = f.readlines()

Splitting Lines

As a last example, we will show how to split lines from a text file.

The split function in our example, splits the string contained in the variable
data, whenever it sees a space character. 

You can split by whatever you wish, line.split(":") would split the line using
colons.
with open('data.txt', 'r') as f:
    data = f.readlines()

    for line in data:
        words = line.split()
        print words
Output:

Because multiple values are returned by split, they are returned as an array.
['hello', 'world,', 'how', 'are', 'you', 'today?']
['today', 'is', 'saturday']

Wednesday, October 12, 2016

[ERROR] No compiler is provided in this environment. Perhaps you are running on a JRE rather than a JDK?

This means you are running jenkins with Open JDK, You have to install Oracle JDK for jenkins to run smoothly.

Follow the instructions in below link
http://nixpoint.blogspot.com/2016/10/confused-between-open-jdk-oracle-jdk.html


Confused between Open JDK & Oracle JDK ?


Variations of Java


There are three different editions of the Java Platform: Standard Edition (SE), Enterprise Edition (EE), and Micro Edition (ME). This tutorial is focused on Java SE (Java Platform, Standard Edition).


There are two different Java SE packages that can be installed: the Java Runtime Environment (JRE) and the Java Development Kit (JDK). JRE is an implementation of the Java Virtual Machine (JVM), which allows you to run compiled Java applications and applets. JDK includes JRE and other software that is required for writing, developing, and compiling Java applications and applets.


There are also two different implementations of Java: OpenJDK and Oracle Java. Both implementations are based largely on the same code but OpenJDK, the reference implementation of Java, is fully open source while Oracle Java contains some proprietary code. Most Java applications will work fine with either but you should use whichever implementation your software calls for.


You may install various versions and releases of Java on a single system, but most people only need one installation. With that in mind, try to only install the version of Java that you need to run or develop your application(s).
If Oracle JDK is installed, output will be like below
[root@Ans.Master-~]# java -version
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)

If it is Open jdk output will be like below

[root@Ans.Master-~]# java -version
java version "1.7.0_15"
OpenJDK Runtime Environment (IcedTea6 1.10pre) (7b15~pre1-0lucid1)
OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)

How to remove OpenJDK and install Oracle JDK


# yum -y remove java*

Download Oracle JDK


wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" \
"http://download.oracle.com/otn-pub/java/jdk/7u79-b15/jdk-7u79-linux-x64.rpm"
rpm -ivh jdk-7u79-linux-x64.rpm

[root@Ans.Master-~]# java -version
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)

Note that java will be installed into /usr/java and establish softlinks as follows:


[root@Ans.Master-~]# ll /usr/bin/java

lrwxrwxrwx. 1 root root 26 Oct 13 01:45 /usr/bin/java -> /usr/java/default/bin/java




Setup Global Environment Variables

We can easily set the environment variables using the export command as shown below.
export JAVA_HOME=/usr/java/jdk1.8.0_25/
export PATH=$PATH:$JAVA_HOME

JAVA_HOME will look for /bin/java so /usr/java/jdk1.8.0_25/ will be right path
But for $PATH we may need to point it to /usr/java/jdk1.8.0_25/bin..not sure if the above step is correct.
If you want JAVA_HOME to be set for every user on the system by default, add the previous line to the/etc/environment file. An easy way to append it to the file is to run this command:

  sudo sh -c "echo export JAVA_HOME=/usr/java/jdk1.7.0_79 >> /etc/environment"
OR
You need to setup global config in /etc/profile OR /etc/bash.bashrc file for all users
If you have multiple java installations in the system you need to make the preferred one as default 



Using Alternatives


The alternatives command, which manages default commands through symbolic links, can be used to select the default Java command.
To print the programs that provide the java command that are managed by alternatives, use this command:
sudo alternatives --config java
Here is an example of the output:
There are 5 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
*+ 1           /usr/java/jdk1.8.0_60/jre/bin/java
   2           /usr/java/jdk1.7.0_79/jre/bin/java


Enter to keep the current selection[+], or type selection number: 

Thursday, September 29, 2016

Building images with a Dockerfile

Our first Dockerfile

# Version: 0.0.1
FROM ubuntu:14.04
MAINTAINER James Turnbull "james@example.com"
RUN apt-get update
RUN apt-get install -y nginx
RUN echo 'Hi, I am in your container' \
       >/usr/share/nginx/html/index.html
EXPOSE 80



The Dockerfile contains a series of instructions paired with arguments. Each instruction, for example FROM, should be in upper-case and be followed by an argument: FROM ubuntu:14.04. Instructions in the Dockerfile are processed from the top down, so you should order them accordingly.
Each instruction adds a new layer to the image and then commits the image.

Building the image from our Dockerfile

$ sudo docker build -t="jamtur01/static_web" .
Sending build context to Docker daemon 2.56 kB
Sending build context to Docker daemon
Step 0 : FROM ubuntu:14.04
---> ba5877dc9bec
Step 1 : MAINTAINER James Turnbull "james@example.com"
---> Running in b8ffa06f9274
---> 4c66c9dcee35

Pushing images to the Docker Hub

We push images to the Docker Hub using the docker push command.
$ sudo docker push jamtur01/static_web
The push refers to a repository [jamtur01/static_web] (len: 1)
Processing checksums
Sending image list
Pushing repository jamtur01/static_web to registry-1.docker.io (1↩
tags)
. . .

SOURCE:TheDockerBook by James Turnbull

Ping not working in Docker Containers

1. Install ping yum install iputils
2. Set the permission correctly
    chmod 4755 /bin/ping
3. If you are getting error - /usr/bin/ping: Operation not permitted
    Do the following
    sudo setcap "cap_net_raw+ep" /usr/bin/ping

Wednesday, September 28, 2016

Install Docker in RHEL/CentOS 7

Docker is supported on Red Hat Enterprise Linux 7. Docker requires a 64-bit installation regardless of your Red Hat version. Docker requires that your kernel must be 3.10 at minimum, which Red Hat 7 runs.


Install with yum

  1. Log into your machine as a user with sudo or root privileges.
  2. Make sure your existing yum packages are up-to-date.
    $ sudo yum update
    
  3. Add the yum repo yourself.
    $ sudo tee /etc/yum.repos.d/docker.repo <<-EOF
    [dockerrepo]
    name=Docker Repository
    baseurl=https://yum.dockerproject.org/repo/main/centos/7
    enabled=1
    gpgcheck=1
    gpgkey=https://yum.dockerproject.org/gpg
    EOF
    
  4. Install the Docker package.
    $ sudo yum install docker-engine 
  5. If you are getting ' Public key not installed' error try installing like below
  6. yum install --nogpgcheck docker-engine

  • Start the Docker daemon.

  1. $ sudo service docker start
          or
    $ sudo systemctl start docker.service
    
  2. Verify docker is installed correctly by running a test image in a container.
  3. $ sudo docker run hello-world
    Unable to find image 'hello-world:latest' locally
        latest: Pulling from hello-world
        a8219747be10: Pull complete
        91c95931e552: Already exists
        hello-world:latest: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security.
        Digest: sha256:aa03e5d0d5553b4c3473e89c8619cf79df368babd1.7.1cf5daeb82aab55838d
        Status: Downloaded newer image for hello-world:latest
        Hello from Docker.
        This message shows that your installation appears to be working correctly.
    
        To generate this message, Docker took the following steps:
         1. The Docker client contacted the Docker daemon.
         2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
                (Assuming it was not already locally available.)
         3. The Docker daemon created a new container from that image which runs the
                executable that produces the output you are currently reading.
         4. The Docker daemon streamed that output to the Docker client, which sent it
                to your terminal.
    
        To try something more ambitious, you can run an Ubuntu container with:
         $ docker run -it ubuntu bash
    
        For more examples and ideas, visit:
         http://docs.docker.com/userguide/
  4. Create a docker group

    The docker daemon binds to a Unix socket instead of a TCP port. By default that Unix socket is owned by the user root and other users can access it with sudo. For this reason, docker daemon always runs as the rootuser.
    To avoid having to use sudo when you use the docker command, create a Unix group called docker and add users to it. When the docker daemon starts, it makes the ownership of the Unix socket read/writable by thedocker group.
    Warning: The docker group is equivalent to the root user; For details on how this impacts security in your system, see Docker Daemon Attack Surface for details.
    To create the docker group and add your user:
    1. Log into your machine as a user with sudo or root privileges.
    2. Create the docker group.
      sudo groupadd docker
    3. Add your user to docker group.
      sudo usermod -aG docker your_username
    4. Log out and log back in.
      This ensures your user is running with the correct permissions.
    5. Verify your work by running docker without sudo.
          $ docker run hello-world
      

    Start the docker daemon at boot

    To ensure Docker starts when you boot your system, do the following:
    $ sudo chkconfig docker on
    
    If you need to add an HTTP Proxy, set a different directory or partition for the Docker runtime files, or make other customizations, read our Systemd article to learn how to customize your Systemd Docker daemon options.

    Uninstall

    You can uninstall the Docker software with yum.
    1. List the package you have installed.
      $ yum list installed | grep docker
      yum list installed | grep docker
      docker-engine.x86_64                1.7.1-0.1.el7@/docker-engine-1.7.1-0.1.el7.x86_64
      
    2. Remove the package.
      $ sudo yum -y remove docker-engine.x86_64
      
      This command does not remove images, containers, volumes, or user created configuration files on your host.
    3. To delete all images, containers, and volumes run the following command:
      $ rm -rf /var/lib/docker
      
    4. Locate and delete any user-created configuration files.