Sunday, 11 January 2015

Freight Forwarding and Logistics - another shameless client promotion!

OK, Here we go with another shameless self-promotion for a client for whom we are doing management of ad marketing and promotion.

T ward is a company of some standing in the shipping world, having been around the business for over 100 years they provide a wealth of knowledge for freight forwarding and logistics for numerous clients around the world. Being a complete luddite and also nosy in the same measure I felt a little out of my depth when it came to knowing what made freight forwarding a service at all! With Callum and Nigel's help I was suddenly propelled into the complex and hitherto completely unknown world of international shipping and logistics.

When you imagine these monstrous cargo ships coming in from China or Japan you think, "ok one of those gazillion containers probably contains my next TV" but the scope of the shipping exercise is massive and I ended up applauding the Freight Forwarding guys as they deal with numerous elements which comprise the whole process of logistics as well, including:

  • Customs clearance
  • Haulage and Transportation
  • Storage
  • Payment of duty
  • Freight and related expenses
The massive undertaking that any Freight Forwarding Company takes on board (no pun intended) is something to be marvelled at - each the general points I highlighted above is simply a drop in the ocean (pun deliberate this time) when it comes to the sub-parts which need to be followed, managed and carefully processed to allow your products to be shipped into and out of the UK.

In summary, if you are looking to ship cargo worldwide, then get in touch with a reputable, experienced and knowledgeable agent such as t ward shipping, they really do know what it's all about, and heaven knows, it's a lot!!

Thursday, 20 November 2014

Sports Retail Management - client branded clothing website

We recently developed a new website for Sports Retail Management, a garment supply company who brand up clothes for all sorts of clients. This was a fun project (note the lack of "air quotes" it was quite a challenge, but one things we passed onto them was a lesson learned from days of yore...

This website is a brand new one for us, working closely with the client we created a combination site which has a public element to it coupled with a fully functional client extranet. The client Sports Retail Management, came to us with a simple remit, allow our local authority clients to log onto the website under their own account and create garment orders. "Is that it?" Well, kind of... It then transpired that some clients have some discounts afforded to them based on the level of purchasing they do.....ok, that's not a problem. Oh and each garment can have several colours and sizes. Yeah that's fine. Except that some sizes have different pricing models.

OK so this project started to look a little more complicated than initially envisaged so we started to look at how the system would work in the hands of the user, we started with a skeleton of the functional elements and imagined (note that) what features might be useful for the Authority users.

Side note: we pride ourselves on (mostly) listening to our clients regarding what they want and adding value to the systems we develop to make them as intuitive and user-friendly as we can. One of the things we have learned is one, don't overthink something - brainstorming something is useful but only come to conclusions if you are sure the end-user will see and use the value of your efforts. A number of times I've seen projects (which we have inherited) with loads of bells and whistles, so many if them in fact that you can't get to grips with the underlying functional purpose of the project. A developer who can see the actual value (rather than perceived value) and then apply this into functionality is worth their weight in gold, every time.

So, back to the story, we sat down with the end user and asked them what they would use. This was an eye opener - it's all too common to hear our client ask for this and that, without really knowing what their client would see as the priority feature.

In this case there was a large degree of overlap, with one or two things which the client in particular wanted which were not mentioned at all during the development specification phase.

So a lesson learned, not in an accusing manger it's just that sometimes you would be best advised speaking with the end user as the client, like you, will only imagine what can be the best features for a site which is heavily used by clients.

There endeth the lesson...

Saturday, 20 September 2014

Doing the backup shuffle - removing backups with PHP and CRONTAB jobs

If you, like me, use backups for MySQL or SQL (I guess we all probably should!) then you can do daily backups of your .sql or .bak databases dumps and put them somewhere where you can access them in the (hopefully lack of) event of a disaster occurring with your database or server.

Now, I run a script (like the one I explained in an earlier post for Scheduled SQL Server Backups ) which like most good ideas for backups, takes them away from the actual server - it never ceases to amaze me how some people consider a backup to the same server as the database as a valid way of performing incremental backups, anyway...so I have the SQL files backed up offsite, fine.

However, the database files I have backed up are becoming quite..chunky in size, one bumping the Gb mark and a few others climbing in size as content and complexity are added. This can cause a problem on the server I save the backups to, as I only pay for a limited amount of disk storage space and to be honest I (and my clients) will be unlikely to want to retrieve or restore a database which is older than a weeks worth of daily backups so I needed a way of easily trimming the list and keeping the backup number at a manageable size.

Getting the plan together


So looking at the specification of what I wanted to do:

  1. list the files I want to examine
  2. identify those older than 5 days
  3. delete them
  4. send me an email to let me know the task ran ok
  5. commit the job to Crontab
OK so first things first. The PHP we needed to use would list the files, look at modified date and remove (unlink in PHP terms) from the folder, easy enough:

$dateminusone = date("Y/m/d", strtotime( '-5 days' ) );
$list = glob('*.zip');
usort(
   $list,
   create_function('$a,$b', 'return filemtime($a) - filemtime($b);')
);
foreach($list as $file)
{
 $filemoddate = date('Y/m/d', filemtime($file));
 deloldfile($file,$filemoddate,$dateminusone);
}
sendconfirmation("me", "
my email address");



breaking it down

$dateminusone = date("Y/m/d", strtotime( '-5 days' ) );

sets a variable to set a time to delete back to:

$list = glob('*.zip');

the PHP5.3 function "glob()" allows us to list files of a certain type in a particular folder

usort(   $list,   create_function('$a,$b', 'return filemtime($a) - filemtime($b);'));

I used the usort element in the script as I initially echoed the list back to the screen, but it was a handy way of listing the items in date ascending order according the modified time functio: filemtime(). After creating the usort array of files we iterated through the list with the foreach() loop and ran an internal function to delete the file

deloldfile($file,$filemoddate,$dateminusone);

If we look at the function deloldfile() in its glory we can see that it does a quick check on the modified date versus the set date and if it's older, it's deleted:

function deloldfile($file,$mdate,$startdate){
if( $startdate>$mdate ){
 unlink($file);
 #echo "marked for deleted";
 } else {
 #echo ":)";
 }
}


Once that's done and the loop is complete, we just ping an email to me to tell me that it ran ok

sendconfirmation()

The function which emails me is a simple little SMTP email script which makes it more likely to be sent through an authenticated email account:

function sendconfirmation($name, $address){
    $email =
'emailaddress;
    $name = 'backup task' ;
    $subject = 'SQL backups deleted successfully' ;
    $content = '<html><body style="font-family:arial, sans-serif; font-size:12px">The SQL backup delete task ran successfully on :'.date("D dS M,Y h:i a",time())."<br />";

require_once('class.phpmailer.php');
$mail   = new PHPMailer();
$mail->IsSMTP();
$mail->Host       = "smtp_mail_server_name";
$mail->SMTPDebug  = 1;
$mail->SMTPAuth   = true;
$mail->Port       = 25; 
$mail->Username   = "smtp_email_address_account_name
";
$mail->Password   = "account_password";  
$mail->SetFrom($email, $name);
$mail->AddReplyTo($email, $name);
$mail->Subject    = $subject;
$mail->MsgHTML($content);
$sendaddress = $address;
$sendname = $name;
$mail->AddAddress($address, $sendname);
 if(!$mail->Send()) {
   echo "Mailer Error: " . $mail->ErrorInfo;
 } else {
 
 }
}


This uses the excellent SMTP PHPMailer library and I've never had any problems with this script.

Placing it all together and we have a working script which runs perfectly, so how can we automate this? We will create the Linux version of a Scheduled Task through a cron job. As my servers run off Plesk and Centos the UI is presented quite nicely and so long as you do the right set up of the task you should have no problems.

Creating the Cron Job


In Plesk the cron jobs are set up as "scheduled tasks" under tools and settings (10 & 11)

Click on Tools and Settings and then Scheduled Tasks and then set one up as the Root user (I've found problems can occur when running system commands as other users)

Click on Add a Scheduled Task and you should see something quite daunting like this completely unhelpful screen:


A personal word here, I've come to conclusion that Parallels and Plesk vendors do not want any old wannabe to access system things like tasks so they seldom offer any advice for setting them up: if yo mess it up then they have no wish to be held to account.

So setting up a cron job needs a little further explanation:

* means ALL THE TIME

that's the main explanation over. Seriously though, if you place * in all the fields for times the task will run EVERY minute until doomsday (or your server dies whichever is sooner) so you need to give some serious consideration to the frequency you want to execute the task.

For my situation, I wanted to run it once a week, around 11.00am on a Saturday so I have this:


This means that a 1 minute pass 11.00 every month on a Saturday my command will be executed.

So the command (my URL is blurred out) is:

/usr/bin/wget -O - -q http://URL_TO _RUN_FROM/deletebackups.php > /dev/null 2>&1

We want to run a command so we'll fire the WGET command line utility to open a fully declared web page ( A word to the wise, I attempted to set up cron jobs which fire a URL type command and have only ever got this to work well with WGET at the command line)

/usr/bin/wget because in the server these command line utilities tend to reside in the usr/bin folder

There are a number of switches which are available to append to the WGET directive, we are using:

-O - -q

-O : we need to tell the script that it will expect a file to have to interact with

-q : we want to run it in quiet mode, we don't really care what the response is as that is captured by the email alert at the end of the script

http://URL_TO _RUN_FROM/deletebackups.php kinda obvious but a fully qualified domain name and executable script

..and finally, your friend:

 > /dev/null 2>&1

If you leave this out and your email address is the root address for your server you will get an email each time this script runs, not a problem if you are running it weekly, but some of my scripts run each 15 mins, I personally can't be bothered with getting emails each time every scheduled task runs!

Add in   > /dev/null 2>&1 and this blocks in default email owner function.

Save that job and let it run!