Character Classes or Character Sets

http://www.regular-expressions.info/charclass.html

 

Note that the only special characters or metacharacters inside a character class are the closing bracket (]), the backslash (\), the caret (^) and the hyphen (-). The usual metacharacters are normal characters inside a character class, and do not need to be escaped by a backslash. To search for a star or plus, use [+*]. Your regex will work fine if you escape the regular metacharacters inside a character class, but doing so significantly reduces readability.

get the result from commands executed in expect

http://www.perlmonks.org/?node_id=58634

If you have ever used Expect to execute commands on a remote system I am sure you have run into the problem of parsing the command output from the output of the Expect methods exp_before(), clear_accum(), etc… . If you send your command to this subroutine it will parse the output for you and return the command output as it would be returned from a backtick execution. Note: This subroutine uses a global Expect object variable which already has an established connection. The subroutine will not take commands that end in & .. for those commands just use $expect->print method.

# Executes a command via a global expect object and
# returns the output of the command.
#
# Arguments:
#       1 – Command     <string variable containing the command string
+>
# Returns:
#       String variable cotaining the output of the specified command.
#

sub expect_execute($) {
$expect->clear_accum();
my $command = shift;
my $x = ”;
my $temp = ”;
if ( $command =~ /;$/ ) {
chop( $command );
}
print $expect “$command | sed ‘s/^/COMMAND_OUT: /g’; echo -n E
+ND_; echo EXPECT\n”;
$expect->expect( 300, -re => ‘^END_EXPECT’ );
my $result = $expect->exp_before();
( my @result ) = split( /\n/, $result );
$result = ”;
foreach $x ( @result ) {
$temp = $x;
if ( chop( $temp ) eq “\r” ) {
chop( $x );
}
if ( $x =~ m/^COMMAND_OUT: / ) {
$temp = substr( $x, 13 );
$result = $result . $temp . “\n”;
}
}
return $result;
}

What the hell is Perl is?

Perl or Practical Extraction and Report Language is described by Larry Wall, Perl’s author, as follows: “Perl is an interpreted language optimized for scanning arbitrary text files, extracting information from those text files, and printing reports based on that information. It’s also a good language for any system management tasks. The language is intended to be practical (easy to use, efficient, complete) rather than beautiful (tiny, elegant, minimal).”

In Unix, how do I use the scp command to securely transfer files between two computers?

Unlike rcp or FTP, scpencrypts both the file and any passwords exchanged so that anyone snooping on the network can’t view them.

Warning: Be careful when copying between hosts files that have the same names; you may accidently overwrite them.

The syntax for the scp command is:

scp [options] [[user@]host1:]filename1 … [[user@]host2:]filename2

[[user@]host1:]filename1 is the source file and path, and [[user@]host2:]filename2 is the destination.

For example, if user dvader is on a computer called empire.gov, and wants to copy a file called file1.txt to a directory called somedir in his account on a computer called deathstar.com, he would enter:

scp file1.txt dvader@deathstar.com:somedir

Likewise, if he wanted to copy the entire contents of the somedir directory on deathstar.com back to his empire.gov account, he would enter:

scp -r dvader@deathstar.com:somedir somedir

Similarly, if he is working on another computer, but wanted to copy a file called file1.txt from his home directory on empire.gov to a directory called somedir in his account on deathstar.com, he would enter:

scp dvader@empire.gov:file1.txt dvader@deathstar.com:somedir When using wildcards (e.g.,  *  and  ? ) to copy multiple files from a remote system, be sure to enclose the filenames in quotes. This is because the Unix shell, not the scp command, expands unquoted wildcards

executing external commands in Perl

There are many ways to execute external commands from Perl. The most commons are:

  • system function
  • exec function
  • backticks (“) operator
  • open function

All of these methods have different behaviour, so you should choose which one to use depending of your particular need. In brief, these are the recommendations:

method use if …
system() you want to execute a command and don’t want to capture its output
exec you don’t want to return to the calling perl script
backticks you want to capture the output of the command
open you want to pipe the command (as input or output) to your script

More detailed explanations of each method follows:

 

  • Using system()

 

system() executes the command specified. It doesn’t capture the output of the command.

system() accepts as argument either a scalar or an array. If the argument is a scalar, system() uses a shell to execute the command (“/bin/sh -c command”); if the argument is an array it executes the command directly, considering the first element of the array as the command name and the remaining array elements as arguments to the command to be executed.

For that reason, it’s highly recommended for efficiency and safety reasons (specially if you’re running a cgi script) that you use an array to pass arguments to system()

Example:

#-- calling 'command' with arguments
system("command arg1 arg2 arg3");

#-- better way of calling the same command
system("command", "arg1", "arg2", "arg3");

The return value is set in $?; this value is the exit status of the command as returned by the ‘wait’ call; to get the real exit status of the command you have to shift right by 8 the value of $? ($? >> 8).

If the value of $? is -1, then the command failed to execute, in that case you may check the value of $! for the reason of the failure.

Example:

system("command", "arg1");
if ( $? == -1 )
{
print "command failed: $!\n";
}
else
{
printf "command exited with value %d", $? >> 8;
}

 

  • Using exec()

 

The exec() function executes the command specified and never returns to the calling program, except in the case of failure because the specified command does not exist AND the exec argument is an array.

Like in system(), is recommended to pass the arguments of the functions as an array.

 

  • Using backticks (“)

 

In this case the command to be executed is surrounded by backticks. The command is executed and the output of the command is returned to the calling script.

In scalar context it returns a single (possibly multiline) string, in list context it returns a list of lines or an empty list if the command failed.

The exit status of the executed command is stored in $? (see system() above for details).

Example:

#-- scalar context
$result = `command arg1 arg2`;

#-- the same command in list context
@result = `command arg2 arg2`;

Notice that the only output captured is STDOUT, to collect messages sent to STDERR you should redirect STDERR to STDOUT

Example:

#-- capture STDERR as well as STDOUT
$result = `command 2>&1`;

 

  • Using open()

 http://www.perlhowto.com/executing_external_commands

Use open() when you want to:

– capture the data of a command (syntax: open(“command |”))

– feed an external command with data generated from the Perl script (syntax: open(“| command”))

Examples:

#-- list the processes running on your system
open(PS,"ps -e -o pid,stime,args |") || die "Failed: $!\n";
while ( <PS> )
{
#-- do something here
}

#-- send an email to user@localhost
open(MAIL, "| /bin/mailx -s test user\@localhost ") || die "mailx failed: $!\n";
print MAIL "This is a test message";

 

*****************************************************

There are several ways to invoke a programm. One of the main differences is in the returning value.

  • system("wc -l");
    Will call the given command and return the return value of that command. This is usually not what you want, because most of the times wc -l will mean that you want to get the number of lines back from that call and not if that call was successful or not.
  • $nol = `wc -l`
    The backticks call the command and return it’s output into the variable (here $nol. In this case this will be what you want.
  • Another way of doing this is to use
    $nol = qx/wc -l/;
    (mnemonic: qx quote execute). I think is just the same as the backquotes (at least I don’t know any difference)
  • Of course there are other ways (exec,fork) that behave different with respect to processes, but I don’t know much about this

Hope this helps…

Grep and Pipe

Piping

For example, suppose we have a text file called applist.txt and we want to find out what lines of it contain the word “desktop”. It’s easy with pipes. We list the contents of the file applist.txt and send the results to grep, which then filters all lines containing the desired word “desktop”, and displays those lines on your screen:
$ cat applist.txt | grep desktop

Note that grep is, like many Linux commands, case sensitive. This means that the above matches only “desktop”, not “Desktop” or “DESKTOP”. With the -i option the search is case insensitive:
$ cat applist.txt | grep -i desktop

What if you’d like to scroll the output of grep? Well, because grep sends its results to standard output, you can just pipe them to less:
$ cat applist.txt | grep -i desktop | less

Of course you can redirect the output of grep to a file, if you want:
$ cat applist.txt | grep -i desktop > desktop.txt

 

4.2.1. What is grep?

grep searches the input files for lines containing a match to a given pattern list. When it finds a match in a line, it copies the line to standard output (by default), or whatever other sort of output you have requested with options.

Though grep expects to do the matching on text, it has no limits on input line length other than available memory, and it can match arbitrary characters within a line. If the final byte of an input file is not a newline, grep silently supplies one. Since newline is also a separator for the list of patterns, there is no way to match newline characters in a text.

Some examples:

cathy ~> grep root /etc/passwd
root:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin

cathy ~> grep -n root /etc/passwd
1:root:x:0:0:root:/root:/bin/bash
12:operator:x:11:0:operator:/root:/sbin/nologin

cathy ~> grep -v bash /etc/passwd | grep -v nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
news:x:9:13:news:/var/spool/news:
mailnull:x:47:47::/var/spool/mqueue:/dev/null
xfs:x:43:43:X Font Server:/etc/X11/fs:/bin/false
rpc:x:32:32:Portmapper RPC user:/:/bin/false
nscd:x:28:28:NSCD Daemon:/:/bin/false
named:x:25:25:Named:/var/named:/bin/false
squid:x:23:23::/var/spool/squid:/dev/null
ldap:x:55:55:LDAP User:/var/lib/ldap:/bin/false
apache:x:48:48:Apache:/var/www:/bin/false

cathy ~> grep -c false /etc/passwd
7

cathy ~> grep -i ps ~/.bash* | grep -v history
/home/cathy/.bashrc:PS1="\[33[1;44m\]$USER is in \w\[33[0m\] "

With the first command, user cathy displays the lines from /etc/passwd containing the string root.

Then she displays the line numbers containing this search string.

With the third command she checks which users are not using bash, but accounts with the nologin shell are not displayed.

Then she counts the number of accounts that have /bin/false as the shell.

The last command displays the lines from all the files in her home directory starting with ~/.bash, excluding matches containing the string history, so as to exclude matches from ~/.bash_history which might contain the same string, in upper or lower cases. Note that the search is for the string “ps”, and not for the command ps.

Now let’s see what else we can do with grep, using regular expressions.

4.2.2. Grep and regular expressions

Note If you are not on Linux
We use GNU grep in these examples, which supports extended regular expressions. GNU grep is the default on Linux systems. If you are working on proprietary systems, check with the -V option which version you are using. GNU grep can be downloaded from http://gnu.org/directory/.

4.2.2.1. Line and word anchors

From the previous example, we now exclusively want to display lines starting with the string “root”:

cathy ~> grep ^root /etc/passwd
root:x:0:0:root:/root:/bin/bash

If we want to see which accounts have no shell assigned whatsoever, we search for lines ending in “:”:

cathy ~> grep :$ /etc/passwd
news:x:9:13:news:/var/spool/news:

To check that PATH is exported in ~/.bashrc, first select “export” lines and then search for lines starting with the string “PATH”, so as not to display MANPATH and other possible paths:

cathy ~> grep export ~/.bashrc | grep '\<PATH'
  export PATH="/bin:/usr/lib/mh:/lib:/usr/bin:/usr/local/bin:/usr/ucb:/usr/dbin:$PATH"

Similarly, \> matches the end of a word.

If you want to find a string that is a separate word (enclosed by spaces), it is better use the -w, as in this example where we are displaying information for the root partition:

cathy ~> grep -w / /etc/fstab
LABEL=/                 /                       ext3    defaults        1 1

If this option is not used, all the lines from the file system table will be displayed.

4.2.2.2. Character classes

A bracket expression is a list of characters enclosed by “[” and “]”. It matches any single character in that list; if the first character of the list is the caret, “^”, then it matches any character NOT in the list. For example, the regular expression “[0123456789]” matches any single digit.

Within a bracket expression, a range expression consists of two characters separated by a hyphen. It matches any single character that sorts between the two characters, inclusive, using the locale’s collating sequence and character set. For example, in the default C locale, “[a-d]” is equivalent to “[abcd]”. Many locales sort characters in dictionary order, and in these locales “[a-d]” is typically not equivalent to “[abcd]”; it might be equivalent to “[aBbCcDd]”, for example. To obtain the traditional interpretation of bracket expressions, you can use the C locale by setting the LC_ALL environment variable to the value “C”.

Finally, certain named classes of characters are predefined within bracket expressions. See the grep man or info pages for more information about these predefined expressions.

cathy ~> grep [yf] /etc/group
sys:x:3:root,bin,adm
tty:x:5:
mail:x:12:mail,postfix
ftp:x:50:
nobody:x:99:
floppy:x:19:
xfs:x:43:
nfsnobody:x:65534:
postfix:x:89:

In the example, all the lines containing either a “y” or “f” character are displayed.

4.2.2.3. Wildcards

Use the “.” for a single character match. If you want to get a list of all five-character English dictionary words starting with “c” and ending in “h” (handy for solving crosswords):

cathy ~> grep '\<c...h\>' /usr/share/dict/words
catch
clash
cloth
coach
couch
cough
crash
crush

If you want to display lines containing the literal dot character, use the -F option to grep.

For matching multiple characters, use the asterisk. This example selects all words starting with “c” and ending in “h” from the system’s dictionary:

cathy ~> grep '\<c.*h\>' /usr/share/dict/words
caliph
cash
catch
cheesecloth
cheetah
--output omitted--

If you want to find the literal asterisk character in a file or output, use single quotes. Cathy in the example below first tries finding the asterisk character in /etc/profile without using quotes, which does not return any lines. Using quotes, output is generated:

cathy ~> grep * /etc/profile

cathy ~> grep '*' /etc/profile
for i in /etc/profile.d/*.sh ; do

Installing Perl modules from CPAN

There are several ways to get Perl modules from CPAN installed on your unix-based system. Keep in mind that there is always more than one way to do it with Perl, and this is no different. Before embarking upon any installation, it’s a good idea to download the module, unzip it and check out the documentation. In general, though, most modules are installed in the same method.
The simplest way to get Perl modules installed is to use the CPAN module itself. If you are the system administrator and want to install the module system-wide, you’ll need to switch to your root user. To fire up the CPAN module, just get to your command line and run this:

 perl -MCPAN -e shell 

If this is the first time you’ve run CPAN, it’s going to ask you a series of questions – in most cases the default answer is fine. Once you find yourself staring at the cpan> command prompt, installing a module is as easy as install MODULE::NAME– for example, to install the HTML::Template module you’d type:

 cpan> install HTML::Template 

CPAN should take it from there and you’ll wind up with the module installed into your Perl library.
Let’s say you’re on your system command line and you just want to install a module as quickly as possible – you can run the Perl CPAN module via command line perl and get it installed in a single line:

 perl -MCPAN -e 'install HTML::Template' 

As I mentioned earlier, it’s always advisable to download a module yourself, especially if you’re having problems installing with CPAN. If you’re on the command line, you can use something like wgetto grab the file. Next you’ll want to unzip it with something like:

 tar -zxvf HTML-Template-2.8.tar.gz 

This will unzip the module into a directory, then you can move in and poke around – look for the README or INSTALL files. In most cases, installing a module by hand is still pretty easy, though (although not as easy as CPAN). Once you’ve switched into the base directory for the module, you should be able to get it installed by typing:

 perl Makefile.PL make make test make install