Personal Blogs

I am bashing away at this but feeling that I missing too much as I go. As I mentioned before, my strategy is to crash through the course work and get myself plenty of headroom for the TMA. I am currently at the start of week 15 and have completed part 16 so I am about 2 weeks ahead. Tomorrow, I will start on question one of TMA 02 having submitted ICMA 44 some time ago. 

I got 55.67 out of a possible 60 for ICMA44 which gives me another result in the low 90s. 92.78% to be precise. I felt that i could have got 100% if I had tried another go round but I can't see the point in doing that when all I need is 30% to pass the module. So far, I have averaged 85.6% for the ICMAs so I am quite pleased. With 72% for TMA01, I am satisfied that my strategy is working.

Now, to part 15. Wow, what a devil of a part. This part concentrates on extracting data from a MongoDB database and mapping the results. I am finding this incomprehensible. The code is complex and arcane in its presentation and use and there is zero explanation of what is going on. As an example, in 15.1 it shows you how to map accidents in the area of North Wales Police. It does this by giving you a code - 60 - which represents this police area. In the assignment bit, it asks you to map accidents in the Hampshire police area with no indication of the code for Hampshire. In the solution, it shows you how to get the code - but we have never seen this before so I am not sure quite how I was supposed to know how to do it. So it goes on... 

Yet again, I decided to skim the solutions and to make sure I knew where to come back to when/if I need to. I have never had to tackle an OU course in this manner before and I hope not to have to again. I am very fortunate in have a long experience in coding object oriented software against SQL and NoSQL databases so, although I am not rigorously trained in the theory, my experience of the practice seems to be getting me through.

Now, to more interesting things. I have been working quite hard - outside of my allocated OU hours - on getting my new Riak database integrated into some of my fun code. Now that I am retired, I can work on my own projects rather than those for other people (well a lot of the time anyway as I still get paid consultancy). I have a project that, for convenience sake, was using CSV files as its data store. I am slowly converting that to use the Riak database I have running on Amazon AWS. I had most of the CRUD working but had a strange thing when trying to delete objects from buckets. I could delete a single object but when I tried to loop around the keys for the bucket, nothing was getting deleted. I ran the single code through Wireshark and compared it to the same call using Curl and couldn't see any difference. It took me hours to realise that I was switching the parameters to the single delete method when calling from the loop so the http call was badly formed. Did I feel stupid - yes, I did. Still it is sorted now and I can get on with the integration proper.

(Wireshark records all TCP/IP traffic down a network connection so you can see what is being sent. Curl is a Unix command line program that formats and sends HTTP calls.)

One last thing that I found - one of my classes has lots of entries and I firstly put them individually into a single bucket but I found that creating and saving - and reading and decoding the JSON was taking too long so I now put all of the individual items in a JSON array and save them in one go. Much quicker!

I finally had a soul searching moment this week. I was working through week 9 notebooks and realised that I was just executing the code and moving on. This made me think about how I was studying and what it meant for the future.  Finally, I posted the following in the Cafe forum.

"Am I unusual?

I have got myself quite ahead so the break week really was that. I am currently starting on Part 9 (more later) and have nearly completed the first TMA. As a result, I was able to get a few other things done. 

I spent two days sorting out a new camera lens. This wouldn't normally have taken that long but it was a bit of a special requirement and I got myself down a blind alley. I do a lot of scale plastic modelling and run the web site for a local club (IPMS Clacton). This requires that I take photos of the models that the members bring along. They only bring them once so I have to get the images right. I was having some trouble with my existing zoom lens so I needed a special lens. I went up to Norwich to Wex and had a long conversation about macro lenses. Having got home with the new lens, I found that it would create difficulties so went back the next day and changed it. The whole story is on my modelling blog.

We went away on Saturday for an overnight stay at a nice hotel in Aldeburgh and then, on Sunday, drove around the north Norfolk coast through Cromer toWells-next-the-sea (what a great name that is!).

Back to the course. As mentioned, I have nearly completed TMA 01. I have the second part of Question 6 to do. In a TMA with 6 questions and with most questions having multiple parts, this last little bit has a worrying 20 marks assigned to it. This is not a question to take lightly. Fortunately, the bulk of it is one liner Python answers. I am, currently, stuck on the last part - hence me doing my blog. Still, I have until 7th April to sort it out.

Although I don't know Python any depth, I am managing between my programming knowledge, the O'Reilly book "Python for Data Analysis" (available from the OU library, I think), and StackOverflow.com I am managing to complete all of the tasks to, I think, a satisfactory standard. We shall see, though, when I get the results of the TMA - gulp!

I am starting on Part 9. I must say that the book that they have us reading isn't telling me much. I did a unit on databases back in 1980. I didn't finish it but I got 3/4 of the way through and ever since then I have been using them. Thus, the content of these current units is just covering what I already know - albeit in greater depth. I am hoping that this means that I can keep ahead. My arthritis catches me out some days and I don't get much done so the buffer that I have is going to be  needed further into the course.

This week is all about spreadsheets and how people manage to mess them up. I have some direct knowledge of the major effect that having a spreadsheet as your main source of information and of using a spreadsheet inappropriately can have so I am very careful whenever I use them. I am a big user of Excel. For my last business, I used it as my main accounts package. This worked for a few reasons particular to me:

1. 24 years in banking and having completed a big proportion of the Institute of Banker's exams so I have a deep knowledge of accounts management
   
2. 30 years experience in programming at "mission critical" levels
3. 30 years of using spreadsheets in a wide variety of situations (starting with Lotus 1-2-3 in 1985!)

These have ensured that, when I create a spreadsheet, I always incorporate cross checking features that highlight any numeric inaccuracies.

I found that the course work took a very short time to complete which has enabled me to concentrate on the TMA. I want to have this assignment nailed before the break coming up. It is my birthday during the break week so I am hoping to enjoy the rest and not to have to play catch-up.

Now to the TMA. I am concerned that I have nothing to judge my written performance against. Most here will have completed, what I know as, foundation courses recently. These will have eased the student into writing TMAs in a gentle way. My last assignment for the OU was written in 1980. Since then, I have spent my time writing specifications, code and manuals - none of which is a help in writing 300 words on ‘Is facial recognition tech really a threat to privacy?’ . I have no idea as to whether what I write is in the right format, worded correctly, is too detailed or too trivial. I know that I have an aversion to using "clever" words so this may be my downfall. 

Anyway, I have completed the written work up to question 3b and the notebook up to question 4a. Some of the notebook questions don't quite correspond to my code output. As an example, there is a comment - 

# A quick inspection shows that we have one spurious row at the end the DataFrame.

# (The original spreadsheet has an entirely blank line after the Clinical Senate rows.)

Well, I have a blank line there in the spreadsheet itself but it doesn't appear in my dataframe! I took the point of view that I would just comment on this and not worry about it. Later, this appears:

# Finally remove the spurious space at the end of the 'One ' column name.

# Enter your code here.

I don't have that space so I have placed a comment in the box saying:

# I don't have a space at the end of the 'One' column name

I am hoping that this won't cost me any points. I have just the data visualisation part of the notebook and questions 5 & 6 to complete before 7th April so I should be OK. Given that we have only just got the course work for week 8 and there is no sign of anything further yet, I guess that I will have to hope that I can stay one week ahead.

A difficult week, this week. I have a server computer here at home. This runs both a web and an e-mail server. For many years, this was run on an old, home made computer which ran like a train. Some time ago, I decided that it needed a new machine so I bought a nice little Lenovo desktop. This had Window 8.1 (Ugh) installed. I put the server software on there and off it ran. I had a BT Business broadband line which provided me with the required static IP address. When we moved, I had to close down the broadband and sign up for a new contract - this was because the business broadband was running on a residential line and BT couldn't move it along with the phone line! As I was going to be out of service for a while (10 days, they told me) I moved the server to my daughter's school (she owns it) as she had a static IP there. It has been there before so I didn't foresee any trouble. Before it went, I upgraded it to Windows 10 as this was a free upgrade and I hated 8.1.

From then on, we had nothing but problems with the server freezing at random intervals. This had happened before but, at home, we could just reboot Now I was beholden to others, apart from the fact that the school was closed at weekend, etc. Finally, after 4 weeks (! 10 days, they said) I brought the server back to our new apartment. As part of the new regime, I decided to upgrade everything so I replaced the hard disk with a 250MB SSD (solid state disk) and installed Microsoft Server 2012, which I got from the OU through Dreamspark). We had some problems with the wireless connection which was important as there was no space in the apartment hallway for the server to be directly connected to the BT Business Hub.

OK, so back to now. I spent the whole of last week messing around with wireless network connectors until finally I got everything sorted on Saturday. This constant effort has played havoc with my schedule. In the middle of this, I had my new desk and chair delivered. This came on Friday and is part of my Disabled Student status. The man came with the new desk, which was delivered as a kit of parts. It seems that the desk top comes in many different lengths so it arrived without fixing holes. The installer had to spend a good while measuring up for my top, which is 1.6 metres in length! The desk is height adjustable so, if I get too stuck sitting down, I can raise it sufficiently to stand up to work! The chair is infinitely adjustable. I had a 15 minute training session on its features. He also installed an under desk device for the server that I mentioned above. This avoids the problem of cables being stretched. There was also a footrest and a desktop iPad holder. Everything went in fine except for the iPad holder which wasn't able to shrink down to hold my iPad mini in portrait and was too narrow to hold it in landscape! I have bought a small metal one from Amazon that works beautifully - so good that we have have bought two more. You can see the desk in two heights in the images below.

This is the replacement iPad holder. I am very impressed.

So, everything seems to be Ok and I can get back to studying. However, the BT Broadband should be between 8.5 and 9.9 Mbps uploading (which is important for an e-mail and web server) but it is running at 1.8Mbps so the engineer is coming on Tuesday to knock out my planned 2 hour study stint then!

Back to the course. I am trying to keep one week ahead and I had a lot of worries about this week just gone thanks to the dramas of the previous week - in terms of hours required. However, I shouldn't have worried as week 5 is all about presentation and is of a major order of magnitude down on practical projects. In fact, I skipped all of the notebooks on the basis that, if I need to do any map presentations then I can come back and check these out. In fact, I found the whole week quite light on the core stuff. As an example, this is an e-mail I sent to my friend in the USA regarding one element of the reading material.

"I am currently studying Part 5 of the course - “Presentation - Telling the story”.
I have read 58 pages out of the 87 in the week’s course book (an eBook). It comes out with the following steps:
0. Employ a data engineer and visualisation designer (!!) - or move straight to step 1 ( you have to smile - my addition)
1. Know your story
2. Choose a basic graphic form
3. Try out the basic forms
4. Adjust the elements
5. let others comment.
Wow, is this what I am dedicating 8 - 10 hours a week to learning?"

Having read the last few pages of the book, I have decided to move on to week 6 without worrying and expecting to look back at the notebooks if I need to.  I am doing a lot of worrying about the TMA as well but the questions that I am worrying about count for 6 marks in total so I am tempted to leave them with my best shot and hope (expect?) to get 2 or three marks for what I have done.  We shall see if this is a stupid assumption or not! However, as I cannot grasp what they are asking for then I see that there isn't much I can do about it for now. Maybe it will be come clearer as the TMA deadline approaches.

Summing up the week - I learned a lot more about network cards than I needed to know; I am fed up with BT as it is now 5 weeks and I still don't have my contractual business line; I am getting more confused over the course as some parts are extremely hard and others are almost trivial (but maybe that is because of my pre-knowedge?).

I am just starting week three of the course and have had to give myself a reality check. I was looking through the forthcoming Tutor Marker Assignment (TMA) and was panicking a bit at what I was seeing.

Let's step back a bit. I have had a difficult week one way and another - lots of pain in my hips so lots of tiredness and lack of concentration. This has caused me to have to force concentrate on the learning part. I have weeks 1 and 2 put to bed. I have the first ICMA (computer marked assignment and more on that later) finished and I was finding it difficult to stick with reading the recommended research papers for week 3. I have never been a fan of using jargon and, having been away from studying so long, I find that the way a lot of academic papers are phrased strike me as opaque for not much reason. First off, I couldn't find the recommended papers in the OU library and then, when I did, I found them impenetrable. This was probably because of my concentration difficulties. Eventually, I sorted out what the week was about but I was still panicky about the TMA.

Remember that "lack of concentration" means that you don't read things and take in the import. Well, it seems that, with the TMA, the questions (from 3 onwards), I was panicking over had a rider in the title that I had missed. How about "You should be able to answer this question when you have completed Parts 1 to 6 of the module"? Suddenly, it dawned on me that I was only up to Part 3! Giant relaxation all round.

Then how about this one in Notebook 0.4 of week 3. "We cannot cover the cases where the user has created sentinel values to capture semantic variations in missing data." I even posted on the forum for some help here! A fine example of using words to demonstrate writer understanding rather than to promote the same in the reader. This came back from the Moderator in an attempt, I think, to clarify the issue: ""We're not going to look at cases where the user has different flag or signal values to denote different sorts of missing data." Sort of... ". Hmmm. Well, I think I get the message but I would have said that the course will ignore cases where special flags have been used to note dirty data rather than the Nan or NULL that we are going to look at.

Enough of my moaning. I finished Part 3 feeling that I will, maybe, have to come back and re-run the reading but that I knew enough to go on. Today, I started tackling Part 4 and have found this much more to my liking.

On a personal basis, I am ringing tomorrow to see what the delivery hopes are for my new electric desk and ergonomic chair. I think that this combination will help greatly with my concentration.

Now, I need to discuss the first ICMA. I find the whole issue of the ICMAs to be quite strange. They want us to do them but they tell us we only have to get 30% in 5 of the seven. They then tell us that we can have up to three goes at each question. From there, it seems that we can have multiple goes at the whole ICMA and, if the score of the latest submitted one is below that already achieved then it is ignored. It seems to be a scheme to let you get to a 100% score - eventually - so why have the 30% - 5 out of 7 - rule? I have done ICMA 41 (the first) just the once and got 75.11 out of 80 which I make 93.9%. This seems to me to be a good score so I will not be going back and having another go. Incidentally, the scores that are shown on my student record, when I add them up, come to 75.11 whilst the same scores on the Student home page are totalled there as 75.09. I can only guess that the actual scores are  calculated to 3 decimal places with only two shown thus giving a rounding error.

Right, I ought to finish as this is a very long post but I thought that I should mention Microsoft Dreamspark. This is a repository of Microsoft development tools and Operating Systems that can be free if you have the right student credentials. I tried signing up on the web site to find that they hadn't heard of the OU. I even gave them proof of my course but nothing happened. I then found out that the OU has a Dreamspark set up managed by them just for OU students. I sent an e-mail to mct-dreamspark@open.ac.uk and very quickly they had me registered. I was able to download a copy of Windows 10 (for my Windows Laptop which only had Windows 7 so no free upgrade) and Windows 2012 Server to replace the Windows 10 Home edition that I have running on my E-mail and Web Server. They even have MSDos 6.22 if you are so inclined. Try it. You could find something that you need.

Getting closer meant that I had to get more serious about learning Python. 

I run a web and e-mail server from home. The e-mail server is for the family domain and there are three web sites that are running on the machine as well. I thought that I had put together a reliable setup when I  purchased a brand new Lenovo desktop box running Windows 8 (which was fairly quickly upgraded to Windows 10). However, the box freezes occasionally. I can't find any reason for this so I just have to manage the situation. I find the the Pingdom service, which is supposed to e-mail me when my server goes down isn't as reliable as I need.

I was casting around for another program to write in Python to extend my knowledge of the language so I thought that I would write my own Ping software. Now a ping is something that gets sent to a web address and reports back if the server at that address responds or not. What my software had to do was to regularly connect to the server and, if the server failed to respond, send me an e-mail. There was a few problems with me doing this in Python.

1. I had no idea how to connect to a web server
2. I had no idea how to send an e-mail
3. I had no idea how to log these actions

So, this was going to test my ability to learn the inner details of Python. It was actually easier than I thought. Python has a rich set of libraries so I included the following in my program:

import smtplib
import requests
import time
import configparser
import os

My next progression towards the course is a giant backward step. We have decided to move apartments and the move takes place on 23rd January. We are only moving from one floor to another (the new apartment is much bigger than the existing one) but, for various reasons, we are to lose our broadband internet connection for about 10 days - which covers the date of the start of the course! I am hoping that I can use my iPhone as a router for my iMac in the meantime. As we have a nice 4G signal here, that shouldn't be a hardship.

Lastly, I have quite severe Arthritis in a lot of my body so I have registered, through the OU, as disabled. Following an assessment of my problems, I have been awarded a grant that will provide me with equipment that will help me in my studies. I have difficulty sitting for too long in one position and find concentrating for long periods a trial due to pain levels.  Yesterday, I had a visit from a representative of an Ergonomics company who has carried out an assessment for a new desk, chair and reading aids. It seems that I am to get a desk that is electrically height adjustable, a fully configurable chair, an iPad holding device and a footrest. All of this is designed to get me into the best possible position for sitting at my iMac for extended periods. It seems to take about 6 weeks for this to happen but it does mean that, soon after the course starts, I will be able to get on with more success. My existing hobby room is way to small to take the new desk along with my other stuff so it is good that I am moving to a bigger apartment as the 2nd bedroom in that one is substantially larger than now. I might even have room for a decent model railway - at last!!!

I have been writing Smalltalk software since about 1990ish, firstly with VSE and then, from around 1995, IBM VisualAge(now Instantiations). This means that I am very embedded in the world of Object Oriented Software development (OO) and with the standard Smalltalk paradigm where everything is an object. Part of that experience means that I think in objects and write new classes for everything. This isn't always the case for people new to OO where they tend to write sequential code until they are forced into making a class. As an example, I used to work in a large banking software company where I was in charge of development of a reporting system that was being written in Java. This was the first Java development for the team and I was the only person that had really been exposed to OO (I am ignoring the fact the programmers had all been using C++ but I am guessing that they still basically wrote C code).

I was asking one of the senior guys (he had been in the company some 15 years) how it was getting on and his comment was "Glyn (one of the senior developers) tells me that I have to create more classes than currently. I can't see why. The software works, doesn't it?". 

OK, back to the present. The course that I am taking involves using the Python programming language for a lot of the data manipulation. There is a Python package called Pandas that has a huge library of tools for this. I did a three week on line course in Data Manipulation which gave an introduction into Pandas but felt that I really needed to get to grips with Python before the course starts at the end of January. I was mainly concerned with the way that OO works in Python as it looked very easy to write sequential code and ignore OO altogether.

Recently, I have been getting back into the groove of studying for 8 hours, or so, a week. If I don't get into the habit now then it will be difficult when the time comes. Hence, I have been carrying out a few tasks. These have included a course on Calculus, the Data Manipulation course mentioned above and a home study course in Latin (which recently morphed into Dutch). The calculus course was very trivial and I should have kept working through the home study book that I got for my birthday back in March. The data course was very good. The Latin/Dutch was hard because I have forgotten how to sit and learn vocabulary ready for the next session (shades of French and Latin back in my school days so many years ago!). I decided to write a testing program for the vocabulary. I did this in Smalltalk and it works very well as a web development.

Now - to learn Python. Why not recreate the vocabulary program in Python. Admittedly, it will be a command line program so nowhere near the Smalltalk one for presentation but functionality is everything in Python, rather than making nice windows etc. I spent the last 4 of my 2 hour sessions slowly working through the steps and ensuring that the Python code was fully object oriented. Well, yesterday, I was able to go through 10 questions of Dutch words and getting feedback on the right and wrong answers. It still has a way to go but it has proved to me that I "can" do this Python thing. I still have 7 weeks until the course starts so I still have plenty of time to get proficient.

The current Python code can be seen HERE. I will update it as I make further changes.

Lastly, the really good news is that I have been accepted for a Disabled Students Allowance with may provide me with a new, ergonomic, chair and desk that I can sit at for longer then 45 minutes at a time. I will report back on that one.