MySQL 5.5 on Ubuntu 12.04 – Job Failed to Start

I migrated a web site from one host to another and as such upgraded to MySQL 5.5 running on Ubuntu 12.04.  Everything was moving along nicely until MySQL started to randomly crash.

Jul 23 07:06:38 domU-12-31-39-02-76-EA kernel: [236645.626405] init: mysql main process (21515) terminated with status 1
Jul 23 07:06:38 domU-12-31-39-02-76-EA kernel: [236645.626453] init: mysql main process ended, respawning
Jul 23 07:06:39 domU-12-31-39-02-76-EA kernel: [236646.617559] init: mysql post-start process (21516) terminated with status 1

Trying to start the process using the service command yields this message:

Job failed to start

I found several sites recommending that I uninstall and re-install mysqld – which I did to no avail.

I finally tried to run mysqld manually from command line and found a bit more of a clue:

120723 7:40:29 [Note] Plugin 'FEDERATED' is disabled.
120723 7:40:29 InnoDB: The InnoDB memory heap is disabled
120723 7:40:29 InnoDB: Mutexes and rw_locks use GCC atomic builtins
120723 7:40:29 InnoDB: Compressed tables use zlib 1.2.3.4
120723 7:40:29 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(137363456 bytes) failed; errno 12
120723 7:40:29 InnoDB: Completed initialization of buffer pool
120723 7:40:29 InnoDB: Fatal error: cannot allocate memory for the buffer pool
120723 7:40:29 [ERROR] Plugin 'InnoDB' init function returned error.
120723 7:40:29 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
120723 7:40:29 [ERROR] Unknown/unsupported storage engine: InnoDB
120723 7:40:29 [ERROR] Aborting
120723 7:40:29 [Note] /usr/sbin/mysqld: Shutdown complete

Aha!  Looks like something with the configuration of Innodb.

Checking some of these messages revealed that the default configuration for innodb_buffer_pool_size is 128M… for my small server this was simply too large.

From here, I simply modified /etc/mysql/my.cnf to include the following statement

innodb_buffer_pool_size = 16M

A much more reasonable number considering the size of my server memory. Now simply start your mysql server and you should be good to go!

Slow Wireless Download Speeds Solved – Disable WMM Support

I was completely frustrated with the speed of my network connection so I decided to do some investigation. I was convinced that Comcast was to blame and even placed a call to their support department to complain. They told me that the network speeds were as promised and everything checked out. They reset the connection and told me there was nothing they could do.

I ran speed tests (over at http://speedtest.net) and found the download speeds to be anywhere from .66Mbps to .99Mbps.

I tried disabling the security to no avail. I tried disabling the firewall and Upnp – again speeds hovered around 1Mbps. Finally, I disabled WMM support and POW! Speeds increased to over 35Mbps. Below is the screenshot of Cisco configuration.

To modify the configuration, login to your router and select Applications & Gaming, then click QoS and click Disable next to the WMM Support option.  WMM or Wireless Multi-Media Extensions were created to enable better performance (Quality of Service) for video and voice traffic but have the unintended effect of destroying wireless performance.

Disable WMM Support to Fix Slow Wireless Speeds

Disable WMM Support to Fix Slow Wireless Speeds

Graceful Sidebar Updated – Version 1.0.14 Released

The Graceful Sidebar is a WordPress plugin that enables you to display custom sidebar content for each of your posts or pages. This update fixes a problem that resulted in deleted sidebars.  As always, please remember to backup your wordpress database prior to installing this (or any) updated plugin.  Check out this guide to backing up your database for additional details on exactly how to do that.

Zen and the Art of Stack Ranking with Excel

Background

Stack Ranking as the name implies is the process of stacking things or people up and applying a 1 to n rating to each.  Unlike a standard performance rating, a ranking implies that no two employees should have the same assigned rank and that when you’re finished, someone will be number one and someone will be on the bottom of the pile.

Stack Ranking the members of your team can be a loathsome task.  Especially when you consider the fact that many companies use this tool as a way to identify and carry out targeted reductions in force.  The issue that many managers face when it comes to stack ranking is that even if you have a solid team of super hero performers, someone will be on the bottom of the list.

Some have argued that there are many problems with the stack ranking system.  And I must admit the arguments against are valid.

Nevertheless, the stack ranking system, when used effectively can provide great insight into the performance of your team and can highlight important focus areas for enablement and development.  One key to ensuring that you are getting an accurate representation of your team and the rank each member assumes is by including the critical data elements by which you will evaluate them.  This means having some good reference material about what each team member’s job description or role entails.  This will be especially helpful when communicating results to employees as well as superiors.

Implementing a Ranking System

Any ranking system should have a core set of components that will enable you to record the scores of your team members as they relate to criteria or attributes.  I’ve chosen Microsoft Excel as the system I’ll use, but there are many systems available that will work.

Below, I have included an image with various highlighted sections.  The following section of this article will explain the various components and offer suggestions for implementing your own ranking system.

Excel Based Stack Ranking Tool

Core components

A – Success Criteria

Every job has a success profile or a set of criterion upon which success will be judged.  If this has not been communicated properly, you should exit this article here and get that done ASAP.  There is no point in attempting to apply a ranking when your team has no idea about what you consider critical success criteria.  For our implementation, we’re going to start with a two-tiered categorization of success criteria.  This just makes communicating the results easier.

To make things easier to communicate, I’ve enabled a two tiered categorization leveraging the ‘merge and center’ feature of Excel to consolidate the second tier of criteria under the first.

B – Criteria Weighting

I am a huge proponent of incorporating a method of expressing weights to each of the criteria upon which you will rank your staff.  Let’s face it, not every attribute is going to be of equal importance.   Is punctuality as important as follow-through or proactivity?  I won’t debate these here but I will argue that they are of differing levels of importance so any tool used to record your teams’ performance should give you the ability to reflect this.

The weights recorded in this section are used to populate a section of the results summary.   The weighted scores and ranking are calculated by multiplying the score entered by the weight for this criteria divided by the maximum possible weight (10 in my example.)  The function for the weighted score is as follows:

=IF(B5>"",SUM(($E5*$E$4)/10,($F5*$F$4)/10,($G5*$G$4)/10,($H5*$H$4)/10,($I5*$I$4)/10,($J5*$J$4)/10,($K5*$K$4)/10,($L5*$L$4)/10,($M5*$M$4)/10,($N5*$N$4)/10,($O5*$O$4)/10,($P5*$P$4)/10,($Q5*$Q$4)/10,($R5*$R$4)/10,($S5*$S$4)/10),"")

If you’re new to Excel, this will probably look like gibberish. Just understand that the letters and numbers refer to cell locations in the worksheet. Where you see a cell letter or number prefaced with a dollar sign ($), that’s used to harden the reference so that as we iterate through the various cells to perform the calculation, we don’t increment that cell reference.

There’s probably a better or more efficient way to calculate this, but I couldn’t figure one out so I did it the hard way by manually copying and pasting the calculation across all of the various criteria columns.

C – Employees

Ok, now we’re getting to some data entry.  Column B contains a list of your employees, or the members of the team you’re going to be reviewing.

D – Title

Entering a title for each team member under review enables us to gain further insight into performance of the team.  With this, we can answer questions such as “How are Sr. Hero’s doing across our criteria in comparison to Heros?”

E – Discipline

Recording a discipline for each employee is optional.  I view it as essential, however because it will give you even more insight into how your team is performing.  Think of discipline as a specialization within your team.  To determine if you’ll find a discipline useful, examine the differentiation between the roles performed by the members of your team.  Are some focusing on a completely separate project or set of customers?  If so, use this to further differentiate them.

F – Scores

Here’s where the tough work begins.  You  must record a score for each employee as to their performance against the specific criteria.  I’m suggesting a straight integer score from 0 to 10.  To enforce accuracy and eliminate errors in reporting results, I’ve created data validation rules that force users to enter a whole number between 0 and 10.

Conditional Formatting Rules for Scores

As you enter the scores, you will notice that there are special conditional formatting rules applied to the data entry area for scores.  These rules give the data entry area additional readability by color coding high scores in green and low scores in red.  As you complete the process of evaluating and recording scores, the summary results and rankings can be viewed in columns T through Y or sections G through L respectively.

Note, that in columns V and Y, you can sort the data in ascending score or rank by weighted or raw values using the up arrow and down arrow buttons.

G – Score

Section G contains the computed raw score values for each of the values you entered in section F (columns E through S.)  Note that this section does not take into consideration the weightings that you applied in section B.  I have implemented another set of conditional formatting rules to help identify the top and bottom performers by score.  The remainder of the computed sections apply these same conditional formatting rules.

The value for this section is computed by simply summing each of the score columns.


=IF(B5>"",SUM(E5:S5),"")

H – Percentage

Section H contains the percentage of total points scored out of the total possible points.  This field is calculated by dividing the total points scored by the total points available from all of the criteria columns.

=IF(B5>"",T5/150,"")

I – Rank

Section I shows the ranking of the employee based on the unweighted, or raw total score. This is a tricky calculation due to the fact that you may encounter multiple employees with the same exact score. A proper ranking system will never provide the same rank for multiple items or employees. We start with Excel’s built-in RANK() function which will provide a numeric ranking of a series or range of data. However, to ensure that we never present multiple similar ranks, we implement a simple test to add a value to any ranks where duplicate scores in column T may appear. This ensures that no duplicates appear in our ranking system.

=IF(B5>"",RANK(T5,T$5:T$24)+COUNTIF($T$5:T5,T5)-1,"")

J – Weighted Score

Section J gives us an opportunity to leverage the weights you specified in section B for each of the criteria. Here, we’re calculating the total score offset by the individual weights for the separate criteria. This is where is gets a bit tricky. Each score gets multiplied by the weight and then divided by total possible points or 10 in this example.

=IF(B5>"",SUM(($E5*$E$4)/10,($F5*$F$4)/10,($G5*$G$4)/10,($H5*$H$4)/10,($I5*$I$4)/10,($J5*$J$4)/10,($K5*$K$4)/10,($L5*$L$4)/10,($M5*$M$4)/10,($N5*$N$4)/10,($O5*$O$4)/10,($P5*$P$4)/10,($Q5*$Q$4)/10,($R5*$R$4)/10,($S5*$S$4)/10),"")

Be sure to utilize the Up Arrows and Down Arrows in sections M and N to sort the employee data by rankings as you complete the data entry.

K – Weighted Percentage

Section K presents the total percentage of weighted score points achieved versus possible. This is the same calculation that we used for the raw percentage but uses the weighted score in column W instead of the raw score.

=IF(B5>"",W5/$W$4,"")

L – Weighted Rank

Section L gives us a view into the weighted ranking of the employee. Obviously, if you weighted each of the criteria the same, the weighted value and the raw values will be the same and thus so too will the rankings be the same.

M – Raw Rank Sort Buttons

The Up Arrow will enable you to sort the employees by ascending rank based on the raw scores and data whereas the Down Arrow will sort by descending rank.  To enable this sorting feature, I’ve created a button and an associated macro.


Sub btnSortUWAscending()
'
' btnSortUWAscending Macro
'
 Range("alldata").Select
 ActiveWorkbook.Worksheets("Sheet1").Sort.SortFields.Clear
 ActiveWorkbook.Worksheets("Sheet1").Sort.SortFields.Add Key:=Range("uwrank") _
 , SortOn:=xlSortOnValues, Order:=xlAscending, DataOption:=xlSortNormal
 With ActiveWorkbook.Worksheets("Sheet1").Sort
 .SetRange Range("alldata")
 .Header = xlGuess
 .MatchCase = False
 .Orientation = xlTopToBottom
 .SortMethod = xlPinYin
 .Apply
 End With
End Sub

You will notice the use of several named ranges.  For example, “alldata” is a named range which refers to =Sheet1!$A$5:$Y$24.  “uwrank” is a named range which refers to all of the values in the raw ranking calculated column.

You can use the Excel Name Manager to modify this if you should modify the spreadsheet significantly.

Named Range “alldata”

N – Weighted Rank Sort Buttons

The Up Arrow will enable you to sort the employees by ascending rank based on the weighted scores and data whereas the Down Arrow will sort by descending rank.

O – Hide / Unhide Detail Data

This button gives you the ability to hide or show the detail data from sections A, B, and F. The following image shows the result of hiding the detail data.

Detail Data Hidden

Use this button as a toggle to alternately show or hide the detail scores and criteria.

Analyzing the Results

As we can see from the summary view above, we have some issues to address.  The first thing you may gravitate toward is the lower end of the stack but be certain that you must pay just as much if not more attention to those consistently scoring highly.  Retaining top performers is of utmost importance.  You need to be certain that you have a plan to keep these top performers challenged, and interested.  The key to this unfortunately is not illuminated by a tool like this.  That will take more time and effort to get to know your team and what gets them motivated to come to work.

Once you’ve gone through the exercise of recording your team’s scores and reviewed the individual rankings, go back through the scores and make sure that you’ve been objective and that the scores are accurate and reflect real performance.  Also, avoid the temptation to craft the results to get a ranking according to a pre-conceived notion about your team.  Be objective and accept the rankings.  These rankings are a tool to improve the performance of your team as well as improving their level of satisfaction.

Summary

So there you have it, a complete example of a stack ranking system built in Microsoft Excel.  I hope you found this tutorial useful.  As with any tool the hard part is really in the implementation and making sure that you create realistic criteria upon which to judge your employees.  Another reminder to make absolutely certain that the criteria upon which you judge is the same that you’ve communicated to your employees.

If you’ve enjoyed this tutorial and would like a copy of the MS Excel Stack Ranking tool, simply click the following to get a free copy.  Keep in mind that this is a “Macro-Enabled” Microsoft Excel spreadsheet and will require you to enable macro content.  Also – this was created and has only been tested with Microsoft Office Excel Version 2010 although the functions and features used should be backward compatible to older versions of Excel.

Use “Tweet and Get It” or download the macro enabled spreadsheet below.

[tweegi-button name=”stack-ranking-tool”]

 

[download id=”15″]

Excel Project Planning Spreadsheet Version 2

The Excel Project Planning spreadsheet has been a handy tool for many of the projects I’ve worked on and I’ve continued to tweak and add features it. Therefore, I thought I’d add this post to talk about some of the enhancements and provide a link to the updated version.

Enhancements from Version 1 to Version 2

Task color coding based on progress against current day

As you can see from the attached image, the Gantt cells are now color coded.

A – Green cells indicate work that was scheduled and has been completed (100%) as indicated in column F.

B – Yellow cells indicate that tasks were scheduled and more than 50% of the task has been completed but the start date is now in the past.

C – Red. Cells are filled with Red when tasks are scheduled to have had work completed but have not yet started and the start date has come and gone.

D – Dark Blue.  As in version 1 of the Excel Project Planning Spreadsheet, cells filled with dark blue indicate work is scheduled to be started and completed on these dates.

Today’s Date – What/If Analysis

I’ve added a field and some conditional formatting noted in the above image by label E – which enable you to analyze the project as if today’s date were progressing beyond the actual date/time when you’re viewing the project planning spreadsheet.

Clicking the spinner in cell C29 – either up or down will increment the value of Today’s Date.  This will cause the conditional formatting in the Gantt area to show you what the project color coding will look like based on current progress on the date provided in cell B29.

This will also highlight another enhancement that I made to make progress against the current date a bit more obvious.  You’ll notice the dates that appear at the top of the Gantt area that are filled with the project dates (eg: 5/19).  These will now appear in a Grey filled font as they drift into the past.

All of these new enhancements have been accomplished using conditional formatting combined with functions.  To view these, select the cells where the formatting appears and from the home tab, click conditional formatting, then manage rules.

Here’s a link to the spreadsheet download.

[download id=”14″]

I hope you enjoy the new features.  Please Tweet, Like or G+ if so!  Also, if you have a feature request, feel free to comment or use the contact page to reach me.

An Excel Project Planning Spreadsheet

* Be sure to check out the updated version with color coding and other enhancements!

Project planning and progress tracking is an essential part of any successful project, no matter the size. Part of ensuring success is communicating progress with project stakeholders. For small projects, simple meetings will suffice, but anything involving more than a couple of tasks and people typically requires some form of formal project documentation. While Microsoft Project is a great tool, not everyone uses it and while they have a “viewer” version, sometimes it just doesn’t seem worth the hassle. You may want to use a Gantter a free Microsoft Project alternative, sometimes I think it’s just easier to use Excel. While it’s not free – it’s certainly popular, easy to use and offers the ability to export and/or save as PDF files for easy sharing.

Excel Project Plan Example

Excel Project Plan Example

This is why I almost always create a project plan using Microsoft Excel when embarking on a project. In this tutorial, I’ll walk through the steps I took to create a sample Excel-based project planning spreadsheet with a nifty gantt project schedule that shows who’s doing what, when and how complete the tasks are.

Features

I’ve included some important features in this example. Most notably is the gantt schedule view of the project which automatically provides a view of who is performing what tasks on what date. The example includes logic to skip around weekends and non-working days as well as a separate column to track progress against tasks or goals.

To start, I’ve taken a screenshot of my example and I’ve broken it down into the basic sections.

The image below shows the various sections needed to create a basic Excel Project Planning Spreadsheet. Each section is labeled A through F.

Excel Project Planning Example

Excel Project Planning Example

Section A – The What

Column A contains a simple list of the tasks you’ll be looking to achieve.  Start simple but try to get this as complete as possible before moving on to the other sections of the plan.

Section B – The Who

Column B in our project planning spreadsheet contains ‘the who’.  Who will be performing or responsible for the tasks in column A?  This can be a comma separated list but keep all references to a single cell.  One thing to consider is whether or not you want to format the cells to wrap – or simply expand the columns so all of the values can be seen.  If you choose to wrap the cell’s contents, note that the schedule section (F) will contain irregular height’s.

Section C – The When

Columns C and D contain the start and the end dates for when the who will complete the what.

Section D – The Length

Column E is our first computed value and will display the working days between the start in column C and the end of the task in column D.  We use the Excel function networkingdays() to compute this value.

=NETWORKDAYS($C4,$D4)

Section E – Progress (In Percent)

Showing progress is a critical part of communication during any project.  Using Excel’s Conditional Formatting, we can get pretty flashy.

The image below shows the sequence of steps to follow in order to conditionally format column F as data bars representing the total percentage complete for the task in column A.  You’ll want to make sure you select the cells in column F and format them as %, then follow the steps in the image below.

Step 1. Select the Cells in Column F that will be formatted.

Step 2. Make sure you have the “Home” tab active and click the Conditional Formatting Dropdown

Step 3. Select the Data Bars Option and

Step 4. Select one of the Gradient Fill options.

Section F – The Gantt Chart (Visual Schedule)

This is one of the coolest parts of this tool.  Having the ability to visually represent who will be doing what at what specific date is extremely powerful.  To accomplish this, we could simply fill out the project dates in Row 1 starting with column H and manually select the cells that represent the active work schedule for each row using a manual color fill but that would be silly, since we have the data and Excel gives us the ability to conditionally format the cells to represent when active work will be taking place throughout the schedule.

We will start by providing the headings for this section in Row 1 starting in Column H.  Type the first date you expect work to begin into Row 1, Column H.

Excel Format as Month/Day

With H1 selected hover the mouse over the bottom right corner of that cell and the cursor should change to a cross shape.  Clicking the bottom right corner when the cursor shows the thinner

cross shape and dragging horizontally across row one will fill in the sequential dates for the project schedule.  Make sure you go back and select these dates and format them as Month/Day.

Now, we can move on to the tricky part where we tell Excel to color fill only those cells associated with days where work will be performed.  To do this, let’s start by highlighting the entire schedule area where the color fills will appear.  Next, we will select Conditional Formatting and select New Rule.

Select “Use a formula to determine which cells to format”.  This will give us the opportunity to provide a custom formula which will let us leverage the start date in column C, the end date in column D as well as the schedule date in Row 1 Columns H onward to programatically determine which cell should be formatted to contain some color to indicate that work will be performed on this scheduled date.  The following is the formula I’m using to make this determination.

=AND(H$1>=$C3,H$1<=$D3, WEEKDAY(H$1)<7,WEEKDAY(H$1)>1,NOT(NOT(ISNA(VLOOKUP(H$1,'Non-Working Days'!$A$2:$A$7,0,FALSE)))),$F3=1)

Let’s start from left to right and examine how this formula does it’s magic. The first thing to know about conditional formatting is that we must craft a formula that evaluates to true. If the formula evaluates to true, then Excel will apply the formatting rules we specify… otherwise it will leave the cell unformatted.

We start with an ANDfunction. This provides us with the ability to specify multiple conditions, each must evaluate to true. Next, we provide two test conditions:

H$1>=$C3, H$1<=$D3

This test condition asks the question is the value of H1 greater than or equal to the value of cell C3. The dollar sign simply locks us to the first row when we continue evaluating each subsequent cell. Since we know that Row 1, Column H starts with the first scheduled project date and Row 3, Column C contains the start date for the task in row 3, we’re basically asking the question does the heading date in Row 1 column H fall between the start date and the end date for the task in row 3. If so – we’re on our way to understanding that something is going to happen on this date and we’ll need to color in the cell.

Next, we want to make sure we’re not making folks work on Saturday or Sunday and as such, we’re not coloring in any of the schedule blocks for non-working days. We accomplish this by comparing the date in Row 1, Column H with the days of the week that are between Sunday (1) and Saturday (7). the Weekday function returns a number from 1 to 7, Sunday being 1 and Saturday being 7.

WEEKDAY(H$1)<7,WEEKDAY(H$1)>1

If all goes well and we’ve passed the previous conditional tests, we know that a) work is scheduled for this date and b) we’re not on a weekend. Now, we will just want to make sure that we’re not working on a date when the client (or we) are not scheduled to work. This may be because of a scheduling conflict or because of a holiday, for example.

I chose to maintain a list of these non-working dates in a separate location – Sheet 2 for example. The formula simply performs a VLOOKUP function to locate the value of the working date in Column H, Row 1 within the list of non-working days that we created in sheet 2. I’m also checking that we don’t return a #N/A value which happens when the VLOOKUP function doesn’t find a specific value.

Lastly, I’m checking in the percentage complete column (F) for the current task to see if the value is equal to 1… which means the task is complete. I’m doing this because I want to color fill the cell with green to indicate the task is complete. We’ll duplicate the entire conditional function and use another color to indicate that work still needs to be completed on this specific project date.

If you’ve done this correctly, it should look something like the following image.
Excel Project Planning Conditional Formatting Gantt Chart
The image below shows the dates represented in the gantt section with color fills – either green for completed work, or blue for work scheduled but still not complete.

From this point, you’re mostly complete. I like to add some finishing touches, like data input validation in the schedule area to prevent users from attempting to put data there; Labels for the weekends and non-working days, etc. We could also automate some more items here – such as adding VBA macros to automate addition of tasks, but this should suffice for starters.

I’ve prepared a sample which you are welcome to download and customize to your liking. Click the following link to download the example used in this post. If you like tutorial, please consider retweeting or liking using the social buttons just under the article title.

[download id=”13″]

Mobile Detection and Redirection – Updated

A user of my Mobile Detection and Redirection script examples dropped by to comment and let me know that the scripts are working – but the mobile sites are not being indexed by google.  This is due to the fact that I neglected to include the Googlebot-Mobile User Agent in the function that detects mobile users and redirects them appropriately.  A quick change to the code and we’re all fixed up… thanks to John!

 

Here’s the updated downloadable code with examples.

[download id=”1″]

Puppet Module List Shows Funny Graphic Characters with PuTTY

Puppet is a configuration management tool that enables you to configure and maintain software and configurations across a wide variety of machines – rather than doing things manually.  Puppet enables you to package software and configurations into something called modules.  You can create these modules by hand or download some freely available modules from the repository available at http://forge.puppetlabs.com.

This post is just a quick note on how to fix a minor, annoying problem encountered when using puppet from the command line over a PuTTY session. The problem can be reproduced using the following command which lists the modules you’ve got installed in your puppet module repository.


[root@puppetmaster ~]# puppet module list
/etc/puppetlabs/puppet/modules
âââ camptocamp-tomcat (v0.0.1)
âââ puppet-wordpress (v0.0.2)
âââ puppetlabs-motd (v1.0.1)
/opt/puppet/share/puppet/modules
âââ puppetlabs-pe_accounts (v1.0.2)
âââ puppetlabs-pe_compliance (v0.0.6)
âââ puppetlabs-pe_mcollective (v0.0.43)
âââ puppetlabs-stdlib (v2.3.1)

Notice the occurance of âââ . These characters are supposed to be translated by PuTTY into sweet looking line graphics. However, the default translation setting for your PuTTY Terminal session is set to Western European.

If we change this setting to UTF-8, we’ll see the characters get translated properly into the line graphics intended by puppet.

Hope this helps!

Create a Simple Document Viewer with ExtJS

A document or report viewer is simply a tool that can be used to provide quick, indexed access to various PDF report files, image files or even word documents. You can easily index your documents as you author them but some versions of PDF tools don’t permit this. Another option would be to create a Microsoft Powerpoint slideshow – but this doesn’t give you the left-hand navigation tree capabilities of an indexed Adobe PDF (for example.)

I wanted to create a tool that could be used to provide visual access to many individual documents while presenting an easy-to-navigation tree structure based on the groupings or categories of these documents.

This article will provide the tools and examples necessary to enable you to create a simple, light-weight, portable (zip and go), cross-browser, web-based (serverless) document viewer. The intent is to have an archive that you can mail or carry on a thumb drive and distribute freely to customers, prospects or colleagues to enable quick and easy viewing and access to a collection of PDF or other files.

As I always do… let’s cut to the chase… looking for the downloads or source code?
[download id=”12″]

Before we start, I’ll also point you to the working demo / example.

A west region with a treeview control and a center panel with details that change as you select items from the treeview. This type of interface is ubiquitous and yet creating something like this to simply view documents may seem very difficult for folks that haven’t worked with javascript. The document viewer application was created to provide a mechanism for viewing collections of sample reports.  A design goal for this application was for it to be completely standalone – not requiring any additional software or web server to view the documents.

At the heart of this system is a perl script – parse_csv.pl. This is a perl script that was written to create a javascript object notation assignment (JSON) used to populate a treeview control for the purpose of displaying example pdf reports with previews. In order to use this package to create your own custom file or report viewer, you must create a comma separated value (CSV) file with the following values.

REPORTFILE,PREVIEW,GROUP,TITLE,DESCRIPTION

Column 1 – Report file specifies the name of the PDF report file that will be downloadable from within the viewer.

Column 2 – Preview file specifies the name of the preview or thumbnail image that will be presented in the details pane of the viewer application.

Column 3 – Group specifies the treeview group where this report will be grouped.  You should sort your csv file so that groups are stored together.

Column 4 – Title specifies the title of the report and will be displayed in the details pane of when the user clicks on the report in the treeview.

Column 5 – Description specifies the details of what the report will show.

Example

REPORTFILE,PREVIEW,GROUP,TITLE,DESCRIPTION
reportfile_number_one.pdf,preview_of_report_file_one.png,Group One, Example of a Report File One,This report shows some serious stuff.
reportfile_number_two.pdf,preview_of_report_file_two.png,Group One, Example of a Report File Two,This report shows some more serious stuff.
reportfile_number_three.pdf,preview_of_report_file_three.png,Group Two, Example of a Report File Three,This report shows some more serious stuff.
reportfile_number_four.pdf,preview_of_report_file_four.png,Group Two, Example of a Report File Four,This report shows some more serious stuff.
reportfile_number_five.pdf,preview_of_report_file_five.png,Group Two, Example of a Report File Five,This report shows some more serious stuff.
reportfile_number_six.pdf,preview_of_report_file_six.png,Group Three, Example of a Report File Six,This report shows some more serious stuff.
reportfile_number_seven.pdf,preview_of_report_file_seven.png,Group Three, Example of a Report File Seven,This report shows some more serious stuff.

You may include the column headers in your file – however, ensure that you use the --skipheader flag when executing the parse_csv.pl script.

Requirements
Perl.  The parse_csv.pl script is written in perl so you will need an environment were you can execute perl scripts.  A subrequirement is that I’m using some libraries that you may not have installed in your perl environment.

Checkout http://search.cpan.org if you get some errors about missing libraries.

Here’s an overview of the process to create your own custom document viewer.

  1. Download the [download id=”12″] package.  This will contain everything you need to

create and customize your own self contained document viewer.  Self contained

means that you don’t even need to host these documents and the viewer on a web

server… you can simply open the index.html file from a browser.

  1. Review the directory tree for the example package.
.
|-- css     Style Sheets
|-- extjs    Javascript Library for the treeview and panel components
|-- images    Various supporting image files (like icons, etc.)
|-- js    Javascript source files
|-- reports    Report directory - this is where you'll place the reports, pdfs and preview thumbnail files that will be displayed in the viewer.
|-- scripts    Scripts used to create and customize your viewer
  1. Since you’ll be creating your own document or report viewer, you’ll probably want to delete the contents of the reports directory.
  1. Create the report files that you’ll be displaying in the document viewer.  Report files can be PDF, PowerPoint (ppt) or Word Document (doc) files.  Typically, the report file examples are multi-page pdf files.  Once you have the report files, copy them into the reports directory in the base of the example package.
  1. Create a screenshot, or preview image that will be displayed in the preview section of the details panel when viewing the report.  zScreen is a great, free utility that lets you capture portions of your screen and save them to files.  I prefer ‘png’ format because of smaller size.  You’ll need to create one preview image for each report you plan to display in the document viewer.  Copy or place these preview image files into the reports directory along side the actual pdf or ppt report files.
  1. Now you’ll need to create a comma-separated file that contains the names of your report files, the groups they’ll be displayed in, the document titles and a description of each.  As previously mentioned the format for this file is REPORTFILE, PREVIEW, GROUP, TITLE, and DESCRIPTION.  You can find an example in the scripts directory in a file called example.csv.
  2. Create the documents.js file.  This is the file which contains the Javascript Notation (JSON) which is used to populate the treeview control as seen in this image.  The package contains a script named parse_csv.pl and can be found in the scripts directory of the example package.  This script should be run from the main directory of the archive and must be provided several command line options in order to execute properly.  The following is an example execution of the script:

scripts/parse_csv.pl --csvfile scripts/example.csv --outputfile js/documents.js

This was executed from the base directory of the example archive file.  The –csvfile argument tells parse_csv.pl where to file the csv file that you created in step 5.  The –outputfile argument tells parse_csv.pl where to write the json.

It is important to note that the script assumes that you have not modified the directory layout and that you are storing your reports and previews or thumbnails in the “reports” directory.  You may run the script with “—help” to obtain additional information and options.

$ scripts/parse_csv.pl --help


Usage: scripts/parse_csv.pl --csvfile <filename> [--outputfile <filename.js>]
[--outputfile <outputfilename>] controls the output javascript filename - defaults to stdout
[--reportdir <reportdirectory>] specifies the directory where the pdf files may be stored
[--thumbdir <thumbnailedirectory>] specifies the directory where the thumbnail or preview image files may be stored
[--htmltitle <htmltitle>] specifies the title to be used in the report viewer html file
[[--overwrite] | [--nooverwrite]] - controls whether the output file will be overwritten

The following is an example of the resultant documents.js file that is created.


var json=[{"iconCls":"group","text":"Group One","children":[{"content":"reports/reportfile_number_one.pdf","description":"This report shows some serious stuff.","size":"16.23 KB","published":"01-29-2012","iconCls":"pdf","preview":"reports/preview_of_report_file_one.png","text":"Example of a Report File One","id":"1","leaf":"true"},{"content":"reports/reportfile_number_two.pdf","description":"This report shows some more serious stuff.","size":"16.23 KB","published":"01-29-2012","iconCls":"pdf","preview":"reports/preview_of_report_file_two.png","text":"Example of a Report File Two","id":"2","leaf":"true"}],"id":0.0960750988024337},{"iconCls":"group","text":"Group Three","children":[{"content":"reports/reportfile_number_six.pdf","description":"This report shows some more serious stuff.","size":"16.23 KB","published":"01-29-2012","iconCls":"pdf","preview":"reports/preview_of_report_file_six.png","text":"Example of a Report File Six","id":"6","leaf":"true"},{"content":"reports/reportfile_number_seven.pdf","description":"This report shows some more serious stuff.","size":"16.23 KB","published":"01-29-2012","iconCls":"pdf","preview":"reports/preview_of_report_file_seven.png","text":"Example of a Report File Seven","id":"7","leaf":"true"}],"id":0.119647523478733},{"iconCls":"group","text":"Group Two","children":[{"content":"reports/reportfile_number_three.pdf","description":"This report shows some more serious stuff.","size":"16.23 KB","published":"01-29-2012","iconCls":"pdf","preview":"reports/preview_of_report_file_three.png","text":"Example of a Report File Three","id":"3","leaf":"true"},{"content":"reports/reportfile_number_four.pdf","description":"This report shows some more serious stuff.","size":"16.23 KB","published":"01-29-2012","iconCls":"pdf","preview":"reports/preview_of_report_file_four.png","text":"Example of a Report File Four","id":"4","leaf":"true"},{"content":"reports/reportfile_number_five.pdf","description":"This report shows some more serious stuff.","size":"16.23 KB","published":"01-29-2012","iconCls":"pdf","preview":"reports/preview_of_report_file_five.png","text":"Example of a Report File Five","id":"5","leaf":"true"}],"id":0.423250772097287}]

var htmltitle='Example Report Viewer;'

Notice that the script determines the proper filetype and assigns the appropriate icon class – pdf for pdfs, ppt for ppts, etc.  Also note that the script determines the file size and date created.  These attributes are displayed in the details panel when viewing the documents.

Again… here’s the goods:
[download id=”12″]