OU blog

Personal Blogs

David Pennington

ICMA45 and TMA02 - thoughts (but no solutions)

Visible to anyone in the world
Edited by David Pennington, Thursday, 26 May 2016, 14:29

I decided that I had to go back to the MongoDB part as, although I was now conversant with Riak, I didn't know enough about MongoDB to tackle the assignments. In the process, I installed on my Mac (using Brew) and found, to my surprise, that it worked first time! I was then able to play with the code in the "Definitive Guide" book. I then went back to the Notebooks and got confused - again - as it appears that this uses its own MongoDB database. I was expecting it to use the one I installed if it was there listening on the right port. I forgot that the Notebook is running inside Vagrant so couldn't see the other instance. Having messed about like this, I decided to finish off the ICMA - which contains questions that require coding to answer!

I answered all of the usual questions and got 29% - I need 30% in 5 ICMAs so if I could get a few more marks, I could put this to bed and stop worrying about ICMAs for the rest of the course. I went through the coding questions and came to one where they were looking for the month with the least accidents. I guessed a month and found that it was right! 10 marks! I ended up with 51.66% so quickly submitted it and now I am ICMA free.

I have now completed Question 1 of TMA 02 so I have plenty of headroom for the rest of the assignment as it isn't due until 16th June.

To bore you with more of my Riak activity, I have been using my son-in-law's expertise to play with the interface from my Smalltalk code into the database. Riak has an HTTP interface, which is what I am using. It also has a direct socket access which requires converting all the data into hex and so on. It has one advantage in that, once connected, Riak keeps the socket open. This should, in theory, provide a much faster interface. However, I must be doing something wrong because, as we now have the GET operation coded using both HTTP and the Protocol Buffers, I did some timings and found that my HTTP interface was almost twice as fast as the direct socket version. I think that we have to examine what our socket stuff is doing as it should be the other way round.

Lastly, I was having problems using JSON as the data format - the JSON interface that I was using comes from the Squeak world and I had to recode it to run under my VisualAge Smalltalk. In the end, I decided that the idea that Riak is a JSON database is a bit of a misnomer as it will accept anything that you give it. I decided to rework the data formats and save all of my objects in CSV format. I already had all the code to do this as it was CSV based before I started with Riak. Anyway, it is all converted and my software is running perfectly! I can now put the Riak stuff to bed for now and, hopefully, pick it up again when I propose my project at the end of the degree (12 months away).

Permalink Add your comment
Share post
David Pennington

Getting more concerned as time goes on + news of Raik

Visible to anyone in the world

I am bashing away at this but feeling that I missing too much as I go. As I mentioned before, my strategy is to crash through the course work and get myself plenty of headroom for the TMA. I am currently at the start of week 15 and have completed part 16 so I am about 2 weeks ahead. Tomorrow, I will start on question one of TMA 02 having submitted ICMA 44 some time ago. 

I got 55.67 out of a possible 60 for ICMA44 which gives me another result in the low 90s. 92.78% to be precise. I felt that i could have got 100% if I had tried another go round but I can't see the point in doing that when all I need is 30% to pass the module. So far, I have averaged 85.6% for the ICMAs so I am quite pleased. With 72% for TMA01, I am satisfied that my strategy is working.

Now, to part 15. Wow, what a devil of a part. This part concentrates on extracting data from a MongoDB database and mapping the results. I am finding this incomprehensible. The code is complex and arcane in its presentation and use and there is zero explanation of what is going on. As an example, in 15.1 it shows you how to map accidents in the area of North Wales Police. It does this by giving you a code - 60 - which represents this police area. In the assignment bit, it asks you to map accidents in the Hampshire police area with no indication of the code for Hampshire. In the solution, it shows you how to get the code - but we have never seen this before so I am not sure quite how I was supposed to know how to do it. So it goes on... 

Yet again, I decided to skim the solutions and to make sure I knew where to come back to when/if I need to. I have never had to tackle an OU course in this manner before and I hope not to have to again. I am very fortunate in have a long experience in coding object oriented software against SQL and NoSQL databases so, although I am not rigorously trained in the theory, my experience of the practice seems to be getting me through.

Now, to more interesting things. I have been working quite hard - outside of my allocated OU hours - on getting my new Riak database integrated into some of my fun code. Now that I am retired, I can work on my own projects rather than those for other people (well a lot of the time anyway as I still get paid consultancy). I have a project that, for convenience sake, was using CSV files as its data store. I am slowly converting that to use the Riak database I have running on Amazon AWS. I had most of the CRUD working but had a strange thing when trying to delete objects from buckets. I could delete a single object but when I tried to loop around the keys for the bucket, nothing was getting deleted. I ran the single code through Wireshark and compared it to the same call using Curl and couldn't see any difference. It took me hours to realise that I was switching the parameters to the single delete method when calling from the loop so the http call was badly formed. Did I feel stupid - yes, I did. Still it is sorted now and I can get on with the integration proper.

(Wireshark records all TCP/IP traffic down a network connection so you can see what is being sent. Curl is a Unix command line program that formats and sends HTTP calls.)

One last thing that I found - one of my classes has lots of entries and I firstly put them individually into a single bucket but I found that creating and saving - and reading and decoding the JSON was taking too long so I now put all of the individual items in a JSON array and save them in one go. Much quicker!

Permalink Add your comment
Share post

This blog might contain posts that are only visible to logged-in users, or where only logged-in users can comment. If you have an account on the system, please log in for full access.

Total visits to this blog: 34359