Monday 21 November 2011

Cygwin - Get that Linux feeling - on Windows!

Ever get frustrated about the limitations of CMD?

Many companies enforce a SOE installation of Windows to keep all client systems alike. For the company it isusually a good strategy, but for IT professionals it can sometimes be somewhat challanging.

From the day I started working in companies where it wasn't suitable running a *NIX flavour on the PC, I have used Cygwin.

As it states on their homepage (http://cygwin.com/) "Get that Linux feeling - on Windows!"... That's exactly what it does.

Here's a quick guide to get Cygwin installed with X server and xterm.

Installation
  1. Download and run setup.exe from http://cygwin.com/
  2. Follow the on-screen instructions up until the dialog Select Packages.
    To find packages, enter the name in the Search box.

    1. First select the package X-start-menu-icons

    2. Then select patchutils

    3. Click Next to install the packages
    4. (If you have 7GB to spare, you could of course just select everything to install)
  3. The next screen will automatically select all dependencies. Just click Next to start the installation phase.
Running Cygwin - XWin Server
During installation, start menu icons is created by default. To use the XWin Server (and xterm), click on the Start icon:
At startup an xterm session is started as well:
 The default look is quite plain and straining on the eye, so I tend to make it a bit more retro as described in the next section...

To start additional xterm sessions, find the X icon next to the clock and click Applications > xterm:



Customisation
At startup, a new xterm window will appear, use this window to patch the system.XWinrc:
  1. Execute the following, which will create a new config file for XTerm:
    echo 'XTerm*Background: black
    XTerm*Foreground: white
    XTerm*toolBar: false
    XTerm*geometry: 160x40' > ~/XTerm
    
As a result, you will end up with a much nicer look and feel:

Sunday 13 November 2011

Runtime log4j configuration

Recently I got struck with the question what log level the rootLogger in log4j was set to.
Normally a quick check of the log4j's property file would give you the answer, but in this case there was a third party application that had multiple ways of defining the log level both at startup and then runtime.

After a quick search I found this blog post: http://nelz.net/2008/04/08/log4j-runtime-configuration/
This will not only show you the runtime value of the loggers but also enable you to change this without require a restart of the application.

In order to rename log4jAdmin.jsp, grab it from the blog and perform the following steps:
Find: "/log4jAdmin.jsp
Replace with: request.getContextPath() + request.getServletPath() + "
Find: log4jAdmin.jsp
Replace with: <%=request.getContextPath() + request.getServletPath() %>

Screenshot:

The JSP with the customisations:
<%@ page language="java" contentType="text/html;charset=UTF-8" %>
<%@ page import="org.apache.log4j.Level" %>
<%@ page import="org.apache.log4j.LogManager" %>
<%@ page import="org.apache.log4j.Logger" %>
<%@ page import="java.util.HashMap" %>
<%@ page import="java.util.Enumeration" %>
<%@ page import="java.util.Set" %>
<%@ page import="java.util.Arrays" %>
<% long beginPageLoadTime = System.currentTimeMillis();%>

<html>
<head>
    <title>Log4J Administration</title>
    <style type="text/css">
        <!--
        #content {
            margin: 0px;
            padding: 0px;
            text-align: center;
            background-color: #ccc;
            border: 1px solid #000;
            width: 100%;

        }

        body {
            position: relative;
            margin: 10px;
            padding: 0px;
            color: #333;
        }

        h1 {
            margin-top: 20px;
            font: 1.5em Verdana, Arial, Helvetica sans-serif;
        }

        h2 {
            margin-top: 10px;
            font: 0.75em Verdana, Arial, Helvetica sans-serif;
            text-align: left;
        }

        a, a:link, a:visited, a:active {
            color: red;
            text-decoration: none;
            text-transform: uppercase;
        }

        table {
            width: 100%;
            background-color: #000;
            padding: 3px;
            border: 0px;
        }

        th {
            font-size: 0.75em;
            background-color: #ccc;
            color: #000;
            padding-left: 5px;
            text-align: center;
            border: 1px solid #ccc;
            white-space: nowrap;
        }

        td {
            font-size: 0.75em;
            background-color: #fff;
            white-space: nowrap;
        }

        td.center {
            font-size: 0.75em;
            background-color: #fff;
            text-align: center;
            white-space: nowrap;
        }

        .filterForm {
            font-size: 0.9em;
            background-color: #000;
            color: #fff;
            padding-left: 5px;
            text-align: left;
            border: 1px solid #000;
            white-space: nowrap;
        }

        .filterText {
            font-size: 0.75em;
            background-color: #fff;
            color: #000;
            text-align: left;
            border: 1px solid #ccc;
            white-space: nowrap;
        }

        .filterButton {
            font-size: 0.75em;
            background-color: #000;
            color: #fff;
            padding-left: 5px;
            padding-right: 5px;
            text-align: center;
            border: 1px solid #ccc;
            width: 100px;
            white-space: nowrap;
        }

        -->
    </style>
</head>
<body onLoad="javascript:document.logFilterForm.logNameFilter.focus();">

<%
    String containsFilter = "Contains";
    String beginsWithFilter = "Begins With";
    String[] logLevels = {"debug", "info", "warn", "error", "fatal", "off"};
    String targetOperation = (String) request.getParameter("operation");
    String targetLogger = (String) request.getParameter("logger");
    String targetLogLevel = (String) request.getParameter("newLogLevel");
    String logNameFilter = (String) request.getParameter("logNameFilter");
    String logNameFilterType = (String) request.getParameter("logNameFilterType");
%>
<div id="content">
<h1>Log4J Administration</h1>

<div class="filterForm">

    <form action="<%=request.getContextPath() + request.getServletPath() %>" name="logFilterForm">
       Filter Loggers:&nbsp;&nbsp;
        <input name="logNameFilter" type="text" size="50" value="<%=(logNameFilter == null ? "":logNameFilter)%>"
               class="filterText"/>
        <input name="logNameFilterType" type="submit" value="<%=beginsWithFilter%>" class="filterButton"/>&nbsp;
        <input name="logNameFilterType" type="submit" value="<%=containsFilter%>" class="filterButton"/>&nbsp;
        <input name="logNameClear" type="button" value="Clear" class="filterButton"
               onmousedown='javascript:document.logFilterForm.logNameFilter.value="";'/>
        <input name="logNameReset" type="reset" value="Reset" class="filterButton"/>
        <param name="operation" value="changeLogLevel"/>
    </form>
</div>

<table cellspacing="1">
    <tr>
        <th width="25%">Logger</th>
        <th width="25%">Parent Logger</th>
        <th width="15%">Effective Level</th>
        <th width="35%">Change Log Level To</th>
    </tr>

    <%
        Enumeration loggers = LogManager.getCurrentLoggers();
        HashMap loggersMap = new HashMap(128);
        Logger rootLogger = LogManager.getRootLogger();

        if (!loggersMap.containsKey(rootLogger.getName())) {
            loggersMap.put(rootLogger.getName(), rootLogger);
        }

        while (loggers.hasMoreElements()) {
            Logger logger = (Logger) loggers.nextElement();
            if (logNameFilter == null || logNameFilter.trim().length() == 0) {
                loggersMap.put(logger.getName(), logger);
            } else if (containsFilter.equals(logNameFilterType)) {
                if (logger.getName().toUpperCase().indexOf(logNameFilter.toUpperCase()) >= 0) {
                    loggersMap.put(logger.getName(), logger);
                }
            } else {
// Either was no filter in IF, contains filter in ELSE IF, or begins with in ELSE
                if (logger.getName().startsWith(logNameFilter)) {

                    loggersMap.put(logger.getName(), logger);
                }
            }
        }
        Set loggerKeys = loggersMap.keySet();
        String[] keys = new String[loggerKeys.size()];
        keys = (String[]) loggerKeys.toArray(keys);
        Arrays.sort(keys, String.CASE_INSENSITIVE_ORDER);
        for (int i = 0; i < keys.length; i++) {
            Logger logger = (Logger) loggersMap.get(keys[i]);

// MUST CHANGE THE LOG LEVEL ON LOGGER BEFORE GENERATING THE LINKS AND THE
// CURRENT LOG LEVEL OR DISABLED LINK WON'T MATCH THE NEWLY CHANGED VALUES
            if ("changeLogLevel".equals(targetOperation) && targetLogger.equals(logger.getName())) {
                Logger selectedLogger = (Logger) loggersMap.get(targetLogger);
                selectedLogger.setLevel(Level.toLevel(targetLogLevel));
            }

            String loggerName = null;
            String loggerEffectiveLevel = null;
            String loggerParent = null;
            if (logger != null) {
                loggerName = logger.getName();
                loggerEffectiveLevel = String.valueOf(logger.getEffectiveLevel());
                loggerParent = (logger.getParent() == null ? null : logger.getParent().getName());
            }
    %>
    <tr>
        <td><%=loggerName%></td>
        <td><%=loggerParent%></td>
        <td><%=loggerEffectiveLevel%></td>
        <td class="center">
            <%
                for (int cnt = 0; cnt < logLevels.length; cnt++) {

                    String url = request.getContextPath() + request.getServletPath() + 
                        "?operation=changeLogLevel&logger=" + loggerName + "&newLogLevel=" + logLevels[cnt] +
                        "&logNameFilter=" + (logNameFilter != null ? logNameFilter : "") +
                        "&logNameFilterType=" + (logNameFilterType != null ? logNameFilterType : "");

                    if (logger.getLevel() == Level.toLevel(logLevels[cnt]) || 
                        logger.getEffectiveLevel() == Level.toLevel(logLevels[cnt])) {

            %>
            [<%=logLevels[cnt].toUpperCase()%>]
            <%
            } else {
            %>
            <a href='<%=url%>'>[<%=logLevels[cnt]%>]</a>&nbsp;
            <%
                    }
                }
            %>
        </td>
    </tr>
    <%
        }
    %>
</table>
<h2>
    Revision: 1.0<br/>
    Page Load Time (Millis): <%=(System.currentTimeMillis() - beginPageLoadTime)%>

</h2>
</div>
</body>
</html>

Saturday 12 November 2011

Automated DB deployment

Automation is one of the key stones in the build and deploy process.
Taking a closer look at dbdeploy (http://dbdeploy.com/) shows us how to automate the database changes.

To illustrate a structured approach using dbdeploy, I have taken the example/build.xml and rewritten it to show how it can be used in an enterprise environment.


Step 1 - Download dbdeploy
Download package from http://code.google.com/p/dbdeploy/downloads/list
Unzip and put build-sysadmin.xml from the bottom of this post in the example directory.


Step 2 - Prepare the database
In the downloaded package, apply the createSchemaVersionTable.<DB>.sql script to the database schemas for which dbdeploy will be used.
This will create the changelog table which dbdeploy uses to keep track of which changes have been applied.

Using the build-sysadmin.xml, execute:
ant -f build-sysadmin.xml init


Step 3 - Setup source repository for deltas
In the source repository, create a new folder where the deltas will be stored.
For each database change, create a new file in sequence, e.g.:
001_create_test_table.sql
002_insert_addresses_into_test_table.sql
003_....

Prepare for using build-sysadmin.xml, create example/TESTDB directory and copy example/*.sql into this directory.


Step 4 - Build script
A common practise is to create artifacts during the build process which are then promoted to the different environments. This script could be as simple as a Ant zip task packaging all files in the delta folder.

Using the build-sysadmin.xml, execute:
ant -f build-sysadmin.xml build
This will create the artifact dist/dbdeploy-artifact.zip


Step 5 - Deploy script
A good approach is to use "update-database-and-apply-as-separate-step" in example/build.xml as a template. This twostep approach gives more control over the changes that have been made in the database.
The build-sysadmin.xml makes use of this, print change and the rollback script before executing them.

Using the build-sysadmin.xml, execute:
ant -f build-sysadmin.xml deploy



More information
See http://code.google.com/p/dbdeploy/w/list for more details on how to use dbdeploy.

build-sysadmin.xml:
<?xml version="1.0" encoding="UTF-8"?>

<project name="dbdeploy_modified_example" default="build">

    <property name="db.driver" value="org.hsqldb.jdbcDriver" />
    <property name="db.url" value="jdbc:hsqldb:file:db/testdb;shutdown=true" />
    <property name="db.user" value="sa" />
    <property name="db.password" value="" />
    <property name="dir.delta" value="TESTDB" />
    <property name="dist" value="dist" />
    <property name="work" value="work" />

    <path id="hsql.classpath">
        <fileset dir=".">
            <include name="hsqldb*.jar"/>
        </fileset>
    </path>

    <path id="dbdeploy.classpath">
        <!-- include the dbdeploy-ant jar -->
        <fileset dir="..">
            <include name="dbdeploy-ant-*.jar"/>
        </fileset>

        <!-- the dbdeploy task also needs the database driver jar on the classpath -->
        <path refid="hsql.classpath" />

    </path>

    <taskdef name="dbdeploy" classname="com.dbdeploy.AntTarget" classpathref="dbdeploy.classpath"/>

    <target name="build" depends="-clean-build-artifact, -build-artifact"/>
    
    <target name="deploy" depends="-clean-deploy-artifact, -deploy-artifact"/>
    
    <target name="init" depends="-clean-build-artifact, -clean-deploy-artifact, drop-and-create-database, create-changelog-table"/>

    <target name="create-changelog-table">
        <sql driver="${db.driver}" url="${db.url}"
             userid="sa" password="" classpathref="hsql.classpath" >
            <fileset file="../scripts/createSchemaVersionTable.hsql.sql"/>
        </sql>
    </target>

    <target name="-clean-build-artifact" description="Cleanup old artifact">
        <delete dir="${dist}"/>
    </target>
    
    <target name="-build-artifact" description="Build an artifact with database changes">
        <mkdir dir="${dist}"/>
        
        <!-- Create artifact with all deltas -->
        <zip destfile="${dist}/dbdeploy-artifact.zip"
            basedir="${dir.delta}"
            includes="*.sql"
        />
    </target>
    
    <target name="-clean-deploy-artifact" description="">
        <delete dir="${work}"/>
    </target>
    
    <target name="-deploy-artifact" description="generate a sql upgrade script">
        <mkdir dir="${work}/deltas"/>
        
        <!-- Unzip deltas to be used for deployment -->
        <unzip src="${dist}/dbdeploy-artifact.zip" dest="${work}/deltas"/>
        
        <!-- use dbdeploy to generate the change script -->
        <dbdeploy driver="${db.driver}" url="${db.url}"
                  userid="${db.user}"
                  password="${db.password}"
                  dir="${work}/deltas"
                  outputfile="${work}/output.sql"
                  undoOutputfile="${work}/undo.sql"
                  dbms="hsql"
                />

        <!-- Print actions to be taken for auditing purposes -->
        <echo message="Executing dbdeploy agains ${db.url} as ${db.user}:"/>
        <loadfile property="outputsql" srcFile="${work}/output.sql"/>
        <echo>${outputsql}</echo>
        
        <!-- Print undo actions for rollback purposes -->
        <echo message="Undo:"/>
        <loadfile property="undosql" srcFile="${work}/undo.sql"/>
        <echo>${undosql}</echo>
        
        <!-- now apply the changescript to the database -->
        <sql driver="${db.driver}" url="${db.url}"
             userid="sa" password="" classpathref="hsql.classpath">
            <fileset file="${work}/output.sql"/>
        </sql>
    </target>

    <target name="drop-and-create-database">
        <delete dir="db"/>
        <mkdir dir="db"/>
    </target>
</project>

How to change file type in CVS

When working with CVS there might be times where files get added in the wrong format (binary or ascii). To convert between these two there are a couple of commands you can use.

  • Convert from binary to ascii
    • Using CVSNT
      Remove all files and then execute in a shell:
      cvs update -ktkv <FILES>
      cvs commit -fm "Changed format to ascii" <FILES>
    • Using CVS
      cvs admin -kkv <FILES>
      cvs update -A <FILES>
      cvs commit -fm "Changed format to ascii" <FILES>
  • Convert from ascii to binary
    • Using CVSNT
      Remove all files and then execute in a shell:
      cvs update -kb <FILES>
      cvs commit -fm "Changed format to binary" <FILES>
    • Using CVS
      cvs admin -kb <FILES>
      cvs update -A <FILES>
      cvs commit -fm "Changed format to binary" <FILES>

updateCVSRoot.bat

Ever been in a situation where you needed users to update the CVSROOT of their locally checked out source?
It's not always as straight forward as checking out the code again so here's a batch script to recursively update the CVS\Root file with the new CVSROOT.
Synopsis: updateCVSRoot.bat <WORKSPACEDIR> <CVSROOT>

@echo off
REM This script is used to update CVS Root in checked out directory
REM SYNOPSIS: updateCVSRoot.bat <WORKSPACEDIR> <CVSROOT>

REM Check arguments
if "%2" == "" goto MISSINGARGUMENT

REM Check if workspace directory exist
IF NOT EXIST %1 GOTO NOWORKSPACEDIR


REM Setup variables
SET W_DRIVE=%~d1
SET W_PATH=%~dp1

REM Enter drive and directory for the root of CVS checked out directory
%W_DRIVE%
cd %W_PATH%

REM Find all Root files and replace with CVSROOT
for /f "tokens=*" %%a in ('dir Root /b /s') do (
  echo %2> "%%a"
)


echo Done!
exit /b 0

:MISSINGARGUMENT
printf "Usage:  %~n0 <WORKSPACEDIR> <CVSROOT>\n"
printf "  WORKSPACEDIR               Root directory to search for filenames called Root\n"
printf "  CVSROOT                            Fully qualified CVSROOT\n"
printf "\n"
printf "Example: %~n0 c:\\\\cvsdir :sserver:username@cvs.localdomain:/my/cvsroot\n"
printf "\n"
exit /b 1

:NOWORKSPACEDIR
echo ERROR: %1 doesn't exist
echo.
exit /b 1

Welcome to a sysadmin's blog


After many years in the IT industry it is time to start sharing my knowledge and findings, both to myself and others.
Without over-committing, I will try to keep the blog going with frequent updates.

Enjoy