:::: MENU ::::

Obfuscating Email Addresses in Perl

Why Obfuscate and Email Address

I was working on a project recently, and for a username, the email address was displayed. This was the only personal information stored in a database as well. Their username would only be visible on the top-right of the page if they were logged in.

So what’s the problem?

If their session gets hijacked, the attacker then has access to their email address. Many sites and frameworks that power those sites use very lazy methods for tracking sessions. They don’t validate IP addresses, User-Agents, or other unique identifiers to help enforce sessions. If someone was to brute force your session ID, or even steal your cookies and replay them, your password would be pointless.

The Code

Here is an example where $user->email is pulled from a database.

my $user_email = $user->email;
my $replace_substring = substr($user_email, floor(index($user_email, "\@")/2), index($user_email, "\@"));
$user_email =~ s/$replace_substring/.../s;

// do what you want with $user_email

This will turn email@domain.com into em...@domain.com.


PHP, FPM and PDO ODBC NoSQLGetPrivateProfileString

ODBC, OH My!

I was working on a project which used Vertica, a columnar-based RDBM, which is a pain in the butt to work with on as a sysadmin.

The Error

The following error was displayed when executing a script from nginx/php-fpm. The same script ran fine from the CLI.

[S1000][unixODBC][DSI] The error message NoSQLGetPrivateProfileString could not be found    in the en-US locale. Check that /en-US/ODBCMessages.xml exists.
[ISQL]ERROR: Could not SQLConnect.

php-fpm.conf:

...
env[VERTICAINI] = /opt/vertica/vertica.ini
env[ODBCINI] = /etc/odbc.ini

I was able to see that there was no ini file being read in for ODBC by debugging the php-fpm process with:

sudo nohup strace -p {PIDFPM-WORKER}


Installing Rockstack On Mac

What Is Rock?

Rockstack is a developer environment (DevEnv) tool that makes installing and managing code bases extremely easy. At my new position, we use the tool frequently to reduce the headache and time to maintain code. A rock project could be any type of code with the addition of a .rock.yml file which contains specifics for running your code. It works with PHP, Node, Perl, Ruby and Python.

Some of the benefits include the ability to manage your composer dependencies or launch a test server.

Installation

Before you continue, make sure you have homebrew installed.

Download the rockstack brew taps and Formula’s:

bash -c "$(curl -fsSL https://raw.github.com/rockstack/utils/master/install)"

Then install a runtime (or multiple) of your choice:

brew install rock-runtime-php55

Setting Your Environment

To prevent conflicts with binary names, if you install PHP, you will not be able to execute it from your existing $PATH. You will need to run:

eval $(rock --runtime=<runtime> env)

Example:

eval $(rock --runtime=php55 env)

Now when you run php -i you will get your PHP info output.

Example .rock.yml

This .rock.yml file will set the runtime to php55, use composer to install and update packages and listen on a local HTTP socket of your choice to test locally. The directory structure for this example is as follows:

/project
    |--- .rock.yml
    |--- bootstrap.php
    |--- web/
          |--- index.php

You can stick something basic into index.php.

.rock.yml

runtime: 
    php55

build: 
    exec composer install -v

update: 
    exec composer selfupdate && composer update -v

run: 
    exec php -S "${HTTP_HOST-0.0.0.0}:${HTTP_PORT-80}" -t "web"

run_daemon: 
    exec php -S "${HTTP_HOST-0.0.0.0}:${HTTP_PORT-8000}" -t "web" &

kill_daemon: 
    kill -9 $(ps aux | grep "php -S" | grep -v "grep" | awk -F" " '{print $2}')

bootstrap.php

<?php
include_once 'vendor/autoload.php';

echo "My First Rock App";

web/index.php

<?php
include_once '../bootstrap.php';

Summary

These basic steps will get rock running in your Mac with a basic config.

Gotcha’s With Rock perl516

I had an issue installing rock-runtime-perl516. I would get the following:

% brew install rock-runtime-perl516
==> Downloading http://www.cpan.org/src/5.0/perl-5.16.3.tar.bz2
Already downloaded: /Library/Caches/Homebrew/rock-runtime-perl516-5.16.3.tar.bz2
==> ./Configure -des -Dprefix=/usr/local/Cellar/rock-runtime-perl516/5.16.3 -Dvendorprefix=/usr/local/Cellar/rock-runtime-perl516/5.16.3 -Dsiteprefix=/usr/local/
==> make
==> make install
==> curl -LO http://search.cpan.org/CPAN/authors/id/A/AP/APEIRON/local-lib-1.008009.tar.gz
==> tar -xzf local-lib-1.008009.tar.gz
==> perl Makefile.PL
==> make install
==> curl -Lo /usr/local/Cellar/rock-runtime-perl516/5.16.3/bin/cpanm https://raw.github.com/miyagawa/cpanminus/1.6008/cpanm
==> chmod 755 /usr/local/Cellar/rock-runtime-perl516/5.16.3/bin/cpanm
==> cpanm -l local http://backpan.cpan.org/authors/id/T/TO/TOKUHIROM/Test-Requires-0.06.tar.gz http://backpan.cpan.org/authors/id/D/DA/DAGOLDEN/Capture-Tiny-0.21
tar: Error exit delayed from previous errors.
! Failed to unpack App-cpanminus-1.7001.tar.gz: no directory
! Failed to fetch distribution App-cpanminus-1.7001
! Bailing out the installation for carton-v0.9.10. Retry with --prompt or --force.
12 distributions installed

READ THIS: https://github.com/mxcl/homebrew/wiki/troubleshooting
If reporting this issue please do so at (not mxcl/homebrew):

https://github.com/rockstack/homebrew-rock/issues

If you look at the formula for brew, it says CPANM version of 1.6008 will be used, but there’s and issue with extracting version 1.7001.

To correct this, you can do:

% cpan App::cpanminus
% cpanm -l local http://backpan.cpan.org/authors/id/T/TO/TOKUHIROM/Test-Requires-0.06.tar.gz \ 
    http://backpan.cpan.org/authors/id/D/DA/DAGOLDEN/Capture-Tiny-0.21.tar.gz \

http://backpan.cpan.org/authors/id/M/MI/MIYAGAWA/App-cpanminus-1.6008.tar.gz

And then when you re-run brew install rock-runtime-perl516, it will execute successfully.


Quickest Way To Upgrade PHP

Keeping PHP Up To Date

One of the most common reasons for security breaches is out of date software. This could mean applications like WordPress or even the parser and HTTP server behind it like PHP and nginx. Personally, I feel there really is no excuse to let minor and even major releases roll by other than pure laziness or lack of dedication to the project. However, there are times when you start the upgrade process and you either lost your original ./configure string or didn’t realize that your configure options were in php -i or <? phpinfo(); ?>.

Regardless of which category you fall into, this trick will make it easier than pie to manage the upgrade process. All you need to do is download the tarball from php.net and extract it.

One of my co-workers lost their original ./configure string and he was enlightened by the tip below.

The One-liner

You can reference your current PHP configuration options by accessing the php-config binary located within PHP_ROOT/bin.

./configure `php-config --configure-options`

No need to hunt down your bash history, your notes or even php -i.


Testing Remote Syslog Connectivity

Remote Syslog

Many large networks and companies will pass their syslog messages to a remote server. This serves 2 key purposes, saves resources on the local device, and allows them to correlate key performance indicators and events across multiple devices. In smaller environments, this usually is not needed.

One problem with remote syslog is that it uses UDP port 514. Since UDP is an unreliable protocol, many simple socket tests will return Successful, when in actuality connection to the remote socket failed. This can be a nightmare if you are trying to test deployment of syslog to your network or check firewall rules.

The only surefire way to test is to send actual syslogs. This sounds like a great idea, but if you are in a scenario where you don’t want to make the changes live, you’re out of luck. Until now.

The following code is written in Perl which allows you to test syslog connectivity without changing system configuration.

The Source Code

This source uses the Sys::Syslog class which can be found on CPAN: Sys::Syslog.

File: syslogtest

# Includes
use Sys::Syslog; # all except setlogsock(), or: 
use Sys::Syslog qw(:DEFAULT setlogsock); # default set, plus setlogsock() 
use Sys::Syslog qw(:standard :macros); # standard functions, plus macros

# Define Vars
my $host = "10.255.11.3";
my $facility = "LOCAL0";
my $ident = "ZypIO";

# Instruct it to use UDP
setlogsock('udp'); 

# Set Syslog Host IP
$Sys::Syslog::host = $host; 

# Open log
openlog($ident, 'ndelay,pid', $facility); 

# Write to log
syslog('info', 'something happened over here'); 

# You can pass the first arg using $ARGV[0] instead
# syslog('info', $ARGV[0]); 

# Close log
closelog;

For the openlog command, it takes 3 parameters:

  • ndelay: Open the connection immediately (normally, the connection is opened when the first message is logged).
  • nowait: Don’t wait for child processes that may have been created while logging the message. (The GNU C library does not create a child process, so this option has no effect on Linux.)
  • pid: Include PID with each message.

Usage:

You can simply execute the file above. It will pass the message you defined in the syslog statement to the remote server. If you comment that line out, and uncomment the following line, you can pass a message to the file when you execute it, for better interoperability.

Example:

./syslogtest "Here is an example message"

This would send the message, “Here is an example message” to the remote syslog server.

Here is a copy of a tcpdump showing the packet getting sent:

2013-07-11 13:26:16.507048 IP (tos 0x0, ttl 64, id 47491, offset 0, flags [DF], proto UDP (17), length 28)
    10.211.55.3.58389 > 10.255.11.3.514: [bad udp cksum 0x57f1 -> 0xc1ee!] [|syslog]
    0x0000:  4500 001c b983 4000 4011 2976 0ad3 3703  E.....@.@.)v..7.
    0x0010:  0aff 0b03 e415 0202 0008 57f1            ..........W.
2013-07-11 13:26:16.507550 IP (tos 0x0, ttl 64, id 47492, offset 0, flags [DF], proto UDP (17), length 93)
    10.211.55.3.58389 > 10.255.11.3.514: [bad udp cksum 0x5832 -> 0x2181!] SYSLOG, length: 65
    Facility local0 (16), Severity info (6)
    Msg: Jul 11 13:26:16 ZypIO[25377]: something happened over here\0x0a\0x00
    0x0000:  3c31 3334 3e4a 756c 2031 3120 3133 3a32
    0x0010:  363a 3136 205a 7970 494f 5b32 3533 3737
    0x0020:  5d3a 2073 6f6d 6574 6869 6e67 2068 6170
    0x0030:  7065 6e65 6420 6f76 6572 2068 6572 650a
    0x0040:  00
    0x0000:  4500 005d b984 4000 4011 2934 0ad3 3703  E..]..@.@.)4..7.
    0x0010:  0aff 0b03 e415 0202 0049 5832 3c31 3334  .........IX2<134
    0x0020:  3e4a 756c 2031 3120 3133 3a32 363a 3136  >Jul.11.13:26:16
    0x0030:  205a 7970 494f 5b32 3533 3737 5d3a 2073  .ZypIO[25377]:.s
    0x0040:  6f6d 6574 6869 6e67 2068 6170 7065 6e65  omething.happene
    0x0050:  6420 6f76 6572 2068 6572 650a 00         d.over.here..

Summary

This script will save you time and headache, especially if you are in a crunch. It is easily configured and simple to run/execute. It is also the most reliable way to test syslog over UDP.


Managing Nginx With nginxmgr

Making Nginx Easier To Manage

When working with nginx, to help reduce the time required to manually create sites, enable, disable and delete them, we created a utility called, nginxmgr. This project is hosted on GitHub and is licensed under GNU-GPL v2. This project was written in Ruby thanks to it’s quick and agile language syntax and performance footprint. Although we are mainly a PHP shop here, we are extremely open-minded and understand that ever language has their pro’s and con’s, and we felt Ruby was a great choice for this project.

Right now, a simple template is included in the package which will create a basic site for you. You can change the template as needed, or wait for the next version which will allow for multiple included templates for things like SSL, Ruby vs PHP FastCGI’s, and more.

Installation:

Before you install nginxmgr make sure you have the basic requirements. This includes an installed version of Ruby >= 1.9.1, a gem named micro-optparse and git. Once you confirmed these, you can continue on.

This installation example will use /usr/local as the basedir (You can change it to whatever you would prefer):

cd /usr/local
sudo git clone https://github.com/mikemackintosh/nginxmgr
cd nginxmgr
sudo chmod +x nginxmgr
sudo ln -s /usr/local/nginxmgr/nginxmgr /usr/bin

Congratulations! Installation has been completed!

Usage:

Here is a copy of the usage info:

# nginxmgr -h
nginxmgr: by Zyp.io
    -i, --ip *                       set virtual host ip address
    -c, --create []                  create a site supplying the server name string
    -e, --enable                     enable a site supplying the server name string
    -d, --disable                    disable a site supplying the server name string
    -D, --delete                     delete a site supplying the server name string
    -h, --help                       Show this message
    -v, --version                    Print version

To Create a Site

To use the utility, you would run the following command:

splug@dev.ve.zyp.io:~$ sudo nginxmgr -c "www.example.com,example.com,*.example.com"
Creating site: www.example.com
[ Completed ] Site has been created
Enabling site: www.example.com
Testing nginx configuration: nginx.
Restarting nginx: nginx.
[ Completed ] This site has been enabled and nginx reloaded
Thank you and have a nice day!

This would create a directory in /var/www/www.example.com (your site name is the first fqdn that you pass) and skeleton directories of web and logs within in. A template for nginx will also be placed within /etc/nginx/sites-available for your domain.

Note: By default, sites that are newely created are automatically

By default the listening IP address and port in *:80. You can change this by passing option -i ip.ad.dr.ess.

To Enable a Site

This utility will create a symbolic link from the sites-available dir to sites-enabled, run configtest on nginx, and restart nginx:

splug@dev.ve.zyp.io:~$ sudo nginxmgr -e www.example.com
Enabling site: www.example.com
Testing nginx configuration: nginx.
Restarting nginx: nginx.
[ Completed ] This site has been enabled and nginx reloaded
Thank you and have a nice day!

To Disable a Site

This utility will delete the symbolic link from the sites-enabled dir, run configtest on nginx, and restart it:

splug@dev.ve.zyp.io:~$ sudo nginxmgr -d www.example.com
Restarting nginx: nginx.
[ Done! ]
Thank you and have a nice day!

Note: This does not delete any userdata, only the link from sites-available to sites-enabled.

To Delete a Site

The -D, --delete switch will remove your domains configuration from nginx, restart if it is currently active, and move the directory /var/www/domain to /var/www/domain.old

splug@dev.ve.zyp.io:~$ sudo nginxmgr -D www.testing2.com
The www directory for this site has been moved to: '/var/www//www.testing2.com.old'
[ Done! ]
Thank you and have a nice day!

Errors

We have been running this utility for a long time and have yet to experience a bug or an issue. If you happen to run into one, let us know and we will get you straightened out. We have done some basic usage testing on platforms that don’t meet the required dependencies, and have come up with the following gotchas.

If you receive the following error:

/usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load such file -- micro-optparse (LoadError)
    from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /usr/bin/nginxmgr:5:in `<main>'

Make sure you execute sudo gem install micro-optparse. This will install the missing dependency.

Summary

We have been using this utility in production for a few months, and it really makes life easier. Everyone is guilty that when out of desperation of a deadline, you fat finger a setting or miss a ;. nginxmgr helps prevent issues like this in the future.

Like we stated earlier, the next version is slated to receive config templates. We will add some options to enable ssl and webdav by default, and even a way to implement WWWBasicAuthentication.

Stay tuned for updates.


Mac OS X Message of the Day

What is the Message of the Day

Every Linux distribution is unique. One way that they are unique is the shell, and even the message of the day displayed when you login to terminal. Ubuntu 11.10 and higher uses landscape if you are on their server build, OS X shows you the last time you logged in. Some platforms will display a random motivation quote or even show you an ASCII art funny.

By nature, I am CLI driven. I don’t mind using a graphical user interface, but I much prefer working through a terminal window. It is much quicker to type than to drag a mouse, and if you are familiar with scripting, I’m sure you agree.

I went ahead and created a PHP script for OS X Mountain Lion (but it does work on 10.6 and up) that will grab some key variables from the system_profiler command, and assign them to a variable.

Note: I use a mid-2012 Mac Book Pro with Retina, so when you look at the network interfaces, mine may not match yours.

What does my Message of the Day look like?

Here is a screenshot of what my new message of the day looks like:

Message of the Day

I’ve broken it down into different categories:

  • Networking
  • System
  • Battery

There are many options and variables available, such as hard drive space, NIC packets in/packets out, your current display brightness, and more.

Usage

Installation instructions and the source code is available on GitHub: mikemackintosh/HackintoshOSX-MOTD

Essentially, this is a PHP script, which executes the system_profiler command. This command will return output that looks similiar to the below:

Firewall:

    Firewall Settings:

      Mode: Limit incoming connections to specific services and applications
      Services:
          Remote Login (SSH): Allow all connections
          Screen Sharing: Allow all connections
      Applications:
          com.apple.imagent: Allow all connections
          com.parallels.desktop.dispatcher: Allow all connections
      Firewall Logging: Yes
      Stealth Mode: No

Graphics/Displays:

    Intel HD Graphics 4000:

      Chipset Model: Intel HD Graphics 4000
      Type: GPU
      Bus: Built-In
      VRAM (Total): 512 MB
      Vendor: Intel (0x8086)
      Device ID: 0x0166
      Revision ID: 0x0009
      gMux Version: 3.2.19 [3.2.8]
      Displays:
        Color LCD:
          Display Type: LCD
          Resolution: 2880 X 1800
          Retina: Yes
          Pixel Depth: 32-Bit Color (ARGB8888)
          Main Display: Yes
          Mirror: Off
          Online: Yes
          Built-In: Yes
          Connection Type: DisplayPort

    NVIDIA GeForce GT 650M:

      Chipset Model: NVIDIA GeForce GT 650M
      Type: GPU
      Bus: PCIe
      PCIe Lane Width: x8
      VRAM (Total): 1024 MB
      Vendor: NVIDIA (0x10de)
      Device ID: 0x0fd5
      Revision ID: 0x00a2
      ROM Revision: 3688
      gMux Version: 3.2.19 [3.2.8]

Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro10,1
      Processor Name: Intel Core i7
      Processor Speed: 2.7 GHz
      Number of Processors: 1
      Total Number of Cores: 4
      L2 Cache (per Core): 256 KB
      L3 Cache: 6 MB
      Memory: 16 GB
      Boot ROM Version: MBP101.00EE.B03
      SMC Version (system): 2.3f35
      Serial Number (system): ----
      Hardware UUID: ----

The PHP script will the loop through the output and create a variable for each item. You can view a list of the supported variables on the GitHub page as well. It then creates an output similar to a bash script which is received via a pipe. It supports terminal colors for your pleasure.

Note: This script is not optimized, but does the job fairly quickly.


PHP 5.5.0: What’s new?

A Bit of PHP History

PHP was originally developed by Rasmus Lerdorf in 1995, some 18 years ago. It’s acronym stood for a very bold and fitting Personal Home Page. In 1997, PHP 3 was releaseed, followed by a release in 2000 for PHP 4. On July 2004, PHP 5 was launched which added support for extremely desirable features such as PHP Data Objects (PDO), Object-Oriented Programming (OOP) and many performance enhancements.

The PHP 5 train began the current generation of the language. PHP 5.0 replaced the core parser with Zend Engine II. In 2005, PHP 5.1 included major changes to performance and formally added PDO support. The following year, in 2006, PHP added native JSON support in PHP 5.2. Up until this time, there were no major changes to the languages nature. In 2009, PHP 5.3 was launched. Many of the features which have been staged for a PHP 6 release were merged to form PHP 5.3. Up until 2 years ago, PHP 5.3 was not taken seriously by many hosting companies which prevented developer access to many of the new features.

PHP 5.3 included Namespace support, allowing you to re-use function names and classes within your code base. It also added Lambda functions or annonymous functions, closures and the addition/removal of other features. The PHP Group took this major release a step further when they launched PHP 5.4 in 2012.

PHP 5.4 added a native web-server for testing and development, support for Traits, reduced memory footprint and removed some insecure and possibly compromising functions in liue of security. This version of PHP also allowed for alternative syntax for arrays which allows you to write to an array like $array_var = array(); or $array = [];.

On June 20th, 2013, PHP 5.5.0 was tagged in the official PHP GIT repository. This is an amazing step forward towards progress when it comes to stabililty, support and reorganization when it comes to an open-source community project. I would suggest you take a look at the changelog for PHP5.5.0 which illustrates its readiness after release candidate 3 status.

What’s New In 5.5.0

There are a few new functions which include:

  1. array_column()
  2. cli_set_process_title()
  3. cli_get_process_title()
  4. boolval()
  5. password_hash()
  6. password_get_info()
  7. password_needs_rehash()
  8. password_verify()

Using generators and functions within foreach

In the example below, make note of the yield keyword. The yield keyword will return values to the foreach function. Now, we no longer need to create a function or an array and pass it to foreach. We can call the function directly within the call to foreach.

<?php
function xrange($start, $limit, $step = 1) {
    for ($i = $start; $i <= $limit; $i += $step) {
        yield $i;
    }
}

echo 'Single digit odd numbers: ';

/* Note that an array is never created or returned,
 * which saves memory. */ 
foreach (xrange(1, 9, 2) as $number) {
    echo "$number ";
}
?>

Finally, finally added to Exception Catching

If you are familiar with Java and C, you know about Exception Handling. Keywords commonly used are throw, which will throw an exception, catch, which will catch a thrown exception, and try which will execute the enclosed block of code which should be evaludated for exceptions.

Previously, PHP 5.4 and 5.3, both support Exceptions, but neither of them had support for the finally keyword. What this keyword allows you to do is to execute code before leaving your try/catch statements.

Example:

<?php
try{
    if(is_null($var)){
        throw new \Exception("Variable \$var is null");
    }
}
catch( \Exception $e ){
    echo $e->getMessage();
}
finally{
    echo "Time to continue execution";
    $var = "Basic default value";
}

Passwords and bcrypt

The fundementals behind bcrypt were implemented into the PHP 5.5.0 source. This allows for the functions prefixed with password_. You can read more about the passwords in the PHP Manual: Password. This change was greatly needed, but once again only works if you can protect your algorithm and salt.

Here is an example from the PHP Manual:

<?php
echo password_hash("rasmuslerdorf", PASSWORD_DEFAULT)."\n";

$options = [
    'cost' => 7,
    'salt' => 'BCryptRequires22Chrcts',
];
echo password_hash("rasmuslerdorf", PASSWORD_BCRYPT, $options)."\n";
?>

The above returns:

$2y$10$.vGA1O9wmRjrwAVXD98HNOgsNpDczlqm3Jq7KnEd1rVAGv3Fykk1a
$2y$07$BCryptRequires22Chrcte/VlQH0piJtjXl.0t1XkA8pw9dMXTpOq

You can then apply a password_verify() on the above responses:

<?php
// See the password_hash() example to see where this came from.
$hash = '$2y$07$BCryptRequires22Chrcte/VlQH0piJtjXl.0t1XkA8pw9dMXTpOq';

if (password_verify('rasmuslerdorf', $hash)) {
    echo 'Password is valid!';
} else {
    echo 'Invalid password.';
}
?>

This returns:

Password is valid!

Take a look at ircmaxell’s git for a userland implementation of the above.

Accessing your class name

Previously, when you wanted to reference your classes name, you had to execute get_class() or call the constant __CLASS__. There is a new way to access your classname resolution using ClassName::class. This can be helpful when using Namespaces, but personally I have not found a use for this yet.

Zend Optimizer

We have not had a chance to play with Zend Optimizer too much, but you can read up on it on Zend’s GitHub.

In summary