Personal Blogs

I have not found this third week particularly easy. I found that Notebook 04.5 particularly time consuming. I am not sure quite where the authors of the course got their timings from but I thought that the following was a bit optimistic:

"Activity 4.5 Notebook - 30 minutes"! At the end, it presents you with a list of tasks:

• a) Show a count of the number of sales records for each District.
• b) Show a count of the number of sales records for each Team in each District, including the Team and District margin totals.
• c) Show the total sales value for each Team in each District summed over the year.
• d) Show the total sales value for each Team Member in each District over the year, showing the District and Team member margin totals. (Remember you need the team name and salesperson name to identify each person uniquely.)
• e) Show a bar chart of the number of sales each month. 
• f) Show a bar chart of the total sales each month.
• g) Show a scatter plot showing the Item Cost v. the number of Units in each record.
• h) Add a Season column to the DataFrame. For each sale record, the value for Season will be derived from the month: (11,12,1) are Winter, (2,3,4) are Spring, (5,6,7) are Summer, (8,9,10) are Autumn. From the sales in each Season calculate the number, average, maximum, minimum and total sale amount over the season (that is, from all the sales records grouped by season report the number of records, and the average, maximum, minimum and total sales amounts).

These would prove to take me 1 1/2 hours on their own and I never did finish so quite where the 30 minutes for the whole notebook came from, I have no idea. Perhaps one of the course authors timed themselves? Anyway, I think that you would need to be very adept at Python and pandas to achieve the time. 

I left that notebook and tackled the next one - about Regular Expressions. I found this not quite so difficult but it did require some careful thinking. By this time, I was getting quite depressed about my chances of even getting to do TMA01, let alone complete it and continue. Fortunately, there was a post on the forum which set me back on the path of confidence (well - a bit). Someone mentioned that there wasn't an exam at the end so we had no need to have everything welded into our brains and that we could manage throughout by referring back to the course notes. A sign of understanding and relief was heard in my household.

On that basis, I tackled the first part of TMA01 and found that I could manage quite well! So, onward and upwards, as the saying goes.

I have been writing successful software for the last 35 years - some of it in mission critical situations such as bank currency trading rooms and so on. This has been done in Basic, which was taught to me by the OU in M251 An algorithmic approach to computing (in 1977), in UCSD Pascal (which was the OU's next teaching language and, finally, Smalltalk - which was the OU's third teaching language (replaced by Java and now Python to some extent). In all that time I have relied heavily on having documentation to hand as I have never managed to learn by rote all of the syntax of these languages. As an aside, Smalltalk (like pandas) has a very easy syntax but an extremely rich library so I spend some time regularly looking up stuff.

The upshot is, I think, that I should assume that I will do the same with Python, pandas and Regex and not try and have it all in my brain. I have one day left to complete week 4 to maintain my one week buffer so it is eyes down tomorrow for a good stint.

One last comment. Last week, I signed up for TM354 - Software Engineering, which starts in October. I have to say that those who know me from my development background expressed opinions that this would be quite boring given my CV. I felt that it would be a reasonable step to the final project course (TM470). However, it turns out that it has an examination. Ignore what I said above as I would be quite happy to sit this if it wasn't for my stupid arthritis and the problems it brings with continuity of concentration. As a result, I cancelled the application and signed up for TM352 - Web, mobile and cloud technologies. Firstly, this doesn't have an exam component but also it ties in very nicely with my thoughts on a project for TM470. More on this later.

Getting closer meant that I had to get more serious about learning Python. 

I run a web and e-mail server from home. The e-mail server is for the family domain and there are three web sites that are running on the machine as well. I thought that I had put together a reliable setup when I  purchased a brand new Lenovo desktop box running Windows 8 (which was fairly quickly upgraded to Windows 10). However, the box freezes occasionally. I can't find any reason for this so I just have to manage the situation. I find the the Pingdom service, which is supposed to e-mail me when my server goes down isn't as reliable as I need.

I was casting around for another program to write in Python to extend my knowledge of the language so I thought that I would write my own Ping software. Now a ping is something that gets sent to a web address and reports back if the server at that address responds or not. What my software had to do was to regularly connect to the server and, if the server failed to respond, send me an e-mail. There was a few problems with me doing this in Python.

1. I had no idea how to connect to a web server
2. I had no idea how to send an e-mail
3. I had no idea how to log these actions

So, this was going to test my ability to learn the inner details of Python. It was actually easier than I thought. Python has a rich set of libraries so I included the following in my program:

import smtplib
import requests
import time
import configparser
import os

My next progression towards the course is a giant backward step. We have decided to move apartments and the move takes place on 23rd January. We are only moving from one floor to another (the new apartment is much bigger than the existing one) but, for various reasons, we are to lose our broadband internet connection for about 10 days - which covers the date of the start of the course! I am hoping that I can use my iPhone as a router for my iMac in the meantime. As we have a nice 4G signal here, that shouldn't be a hardship.

Lastly, I have quite severe Arthritis in a lot of my body so I have registered, through the OU, as disabled. Following an assessment of my problems, I have been awarded a grant that will provide me with equipment that will help me in my studies. I have difficulty sitting for too long in one position and find concentrating for long periods a trial due to pain levels.  Yesterday, I had a visit from a representative of an Ergonomics company who has carried out an assessment for a new desk, chair and reading aids. It seems that I am to get a desk that is electrically height adjustable, a fully configurable chair, an iPad holding device and a footrest. All of this is designed to get me into the best possible position for sitting at my iMac for extended periods. It seems to take about 6 weeks for this to happen but it does mean that, soon after the course starts, I will be able to get on with more success. My existing hobby room is way to small to take the new desk along with my other stuff so it is good that I am moving to a bigger apartment as the 2nd bedroom in that one is substantially larger than now. I might even have room for a decent model railway - at last!!!

I have been writing Smalltalk software since about 1990ish, firstly with VSE and then, from around 1995, IBM VisualAge(now Instantiations). This means that I am very embedded in the world of Object Oriented Software development (OO) and with the standard Smalltalk paradigm where everything is an object. Part of that experience means that I think in objects and write new classes for everything. This isn't always the case for people new to OO where they tend to write sequential code until they are forced into making a class. As an example, I used to work in a large banking software company where I was in charge of development of a reporting system that was being written in Java. This was the first Java development for the team and I was the only person that had really been exposed to OO (I am ignoring the fact the programmers had all been using C++ but I am guessing that they still basically wrote C code).

I was asking one of the senior guys (he had been in the company some 15 years) how it was getting on and his comment was "Glyn (one of the senior developers) tells me that I have to create more classes than currently. I can't see why. The software works, doesn't it?". 

OK, back to the present. The course that I am taking involves using the Python programming language for a lot of the data manipulation. There is a Python package called Pandas that has a huge library of tools for this. I did a three week on line course in Data Manipulation which gave an introduction into Pandas but felt that I really needed to get to grips with Python before the course starts at the end of January. I was mainly concerned with the way that OO works in Python as it looked very easy to write sequential code and ignore OO altogether.

Recently, I have been getting back into the groove of studying for 8 hours, or so, a week. If I don't get into the habit now then it will be difficult when the time comes. Hence, I have been carrying out a few tasks. These have included a course on Calculus, the Data Manipulation course mentioned above and a home study course in Latin (which recently morphed into Dutch). The calculus course was very trivial and I should have kept working through the home study book that I got for my birthday back in March. The data course was very good. The Latin/Dutch was hard because I have forgotten how to sit and learn vocabulary ready for the next session (shades of French and Latin back in my school days so many years ago!). I decided to write a testing program for the vocabulary. I did this in Smalltalk and it works very well as a web development.

Now - to learn Python. Why not recreate the vocabulary program in Python. Admittedly, it will be a command line program so nowhere near the Smalltalk one for presentation but functionality is everything in Python, rather than making nice windows etc. I spent the last 4 of my 2 hour sessions slowly working through the steps and ensuring that the Python code was fully object oriented. Well, yesterday, I was able to go through 10 questions of Dutch words and getting feedback on the right and wrong answers. It still has a way to go but it has proved to me that I "can" do this Python thing. I still have 7 weeks until the course starts so I still have plenty of time to get proficient.

The current Python code can be seen HERE. I will update it as I make further changes.

Lastly, the really good news is that I have been accepted for a Disabled Students Allowance with may provide me with a new, ergonomic, chair and desk that I can sit at for longer then 45 minutes at a time. I will report back on that one.