Welcome

I think I believe you are talking through your hat.

As we approach the final TMA it is worth reinforcing some important points about the critical analyses (and the various variants of a critical analysis such as critical summaries, compare and contrast, etc) and how to approach them. It is quite likely that you have found these difficult in the previous TMAs, especially if you have not studied much at post-graduate level, but they represent a key skill for studying and analysing information in an critical, objective light and they are well worth persevering with.

I am not going to rehash the information on the course website, but would once again encourage you to read it. My focus will be a little different, and give you some clear don'ts.

Firstly, any sentence that starts with the words "I believe" or "I think" (or indeed any equivalent formulation) and then proceeds to make an assertion without offering any supporting evidence is unlikely to gain any credit. Unsupported opinion is not critical analysis.

You may, conceivably, think that the paper doesn't provide sufficient evidence for a conclusion that it draws. You should state that, say what if any evidence it does provide for or against, what it may have missed and why you consider the evidence insufficient - I may agree, I may disagree (in which I will tell you why) - but putting the words "I think" in there won't add value and detracts from a limited word count.

Secondly, and closely related, be careful with the use of personal experience. Personal experience, for example on a project you have worked on, can be very valuable and indeed we would encourage you to think about the topics in the light of your own experience but you must show that you are aware of the difficulty in drawing firm conclusions from such limited data: a particular technique may not have been successful for you but it is difficult to separate the evaluation of the technique from the project and organisational culture you were working in at the time. In engineering we don't in general have randomised controlled trials of course, so observational evidence is often the best evidence we have, but you need to be aware of the limitations.

Thirdly, and coming on to some more practical points, don't bite off too much. You may have found two or more papers rather than just the one - that is fine but you will need to be very concise and focus on clear points you want to make. You have 500 words: it is better to use a single additional paper, or stick to a clear theme that cuts across two or three, than to pick a mishmash of points that don't fit well into a narrative.

Fourthly be aware of those word limits and stick to them. Don't worry too much about writing style - if it reads well that is great (and I may be less likely to count the words) but content is more important.

Fifthly, and finally (for now),a particular point for those for whom English is not their first language. Machine translation tools such as Google Translate should be used with the utmost care: they are powerful and valuable tools but they can very easily lead you astray.

You should never, ever, submit an analysis where you do not understand fully the text you are submitting: a really good piece written in German, submitted to MT (Machine Translation) for translation into English and the result cut and pasted into your TMA will get very poor marks. Not because the use of MT is frowned upon but because the output will almost certainly be gibberish - unreadable and incomprehensible - even if many of the right words are there.

If you use MT you should try to use it in the other direction: write the point you are trying to make in English and then translate to your native language to see if you are saying what you want to say.

It is far better in any case to produce short, precise sentences - bullet points if necessary - rather than complex prose in which the point you want to make can too easily get lost. Yes, you might not gain full style marks but you will get better marks for content.

Hope this helps you all a little, good luck and contact me if you need anything.

Happy New Year all!

This blog is aimed at students on the current (2010K) presentation of M883 but comments and questions are welcome from anyone.

TMA02 is approaching fast, and I am happy to see there is a good deal of activity in the wiki groups. It is only natural that you are thinking of the collaboration first, but I would urge you to think a little about Question 2 before you rush off to think about your functional requirements in Q3: Q2 gives you valuable context, it is where you develop the mental maps that will lead you to the functional requirements.

In all the TMAs, but maybe most importantly in this one because we are now at the very core of the course, questions 2 and 3 lead you through a part of the requirements' analysis process: here setting the context, finding the business events, developing the business use cases into product use cases and on to functional requirements and then refining that list within the collaborative groups.

If you follow the process you will find that functional requirements flow quite naturally from the business events in which your particular actor is involved. Trying to do it backwards is much more difficult!

Of course, there is an important caveat: this is not a reason for failing to meet the deadlines you have all agreed for yourselves.

It does however provide a natural path for your learning in this area, and will make the TMA seem a little easier!

I've had a couple of questions about DFDs, and this part of Q1 is often a stumbling block so I thought I'd just give you a few pointers.

The key point with DFDs is they are formal documents, and as such they have a formal structure and need to be complete and consistent. I don't have the case study or TMA script available at the moment, so I can only comment in a very general way.

1. make sure you include all appropriate data flows, both into and out of the processes. You will also need to think about what other data you need to be able to select an appropriate itinerary. All data flows should be specific, labelled and consistent with the data store they go to or come from.
2. You could start with what is called a "level 0" dfd for your own benefit: a single process and think about what information the customer must provide and what result they should get back. Then develop it to think about what data the system needs to store and add those flows to and from data stores. Then you go to the next level (level 1 dfd, which is effectively what you are looking for here) and break the process down to its next level constituent parts.
3. The general problems I see on this question are:
   • incomplete and poorly labelled data flows and inappropriately (usually vaguely) named data stores inconsistent with the data flows.
   • Not answering the question, which is usually on a very specific change to the existing diagram: concentrate exclusively on that change and answer that part completely. If in doubt about the scope then state your assumptions (that is actually general advice for all questions!).
   • Poor naming of processes, not clear what is intended (your processes have good names - clear, concise and precise - though without better data flows it is difficult to gauge how appropriate they are).

Hope this helps: you can find some more general information at the links below. .

http://openlearn.open.ac.uk/mod/oucontent/view.php?id=397581&section=4.1

http://learn.open.ac.uk/file.php/6968/%21via/resourcepage/63775995/6968/moddata/resourcepage/Some_important_course_concepts_2.pdf

This is loosely based on the question in TMA02.

This basically works for any quality, so I am using reliability as an example.

Let's suppose that we have a system (not yet a product) in which a customer submits a "form" and this must be checked by the work (for completeness and correctness) before being acted on. It is not important in this analysis whether the form is a handwritten form interpreted by a human or an html form interpreted by a computer system: the principles are the same.

Our client has told us that it is essential that the failure rate be low. Naturally we ask what she means by "low" and are told that no more than 1 faulty form in 10 000 should get through.

We still have to ask for further clarification: what is an acceptable false positive failure rate: i.e. the number of correct forms that are rejected. However for simplicity I am going to leave this out.

So we in fact have two requirements: one functional specifying that the form must be complete and correct and one non-functional specifying the reliability quality associated with the test.

For the functional requirement we need to specify exactly what constitutes a failure, so that this is clear and unambiguous.

Once we've done that we can move on to the reliability requirement, and how to specify a fit criterion for this. On the face of it we might say:

The system shall reject at least 9999 out of every 10000 forms that are faulty according to [associated functional requirement].

The first point to note is that we have not yet got enough information to test this and this means the above is not a clear fit criterion. Yes, the required maximum failure rate has been specified but not the tolerance.

If we state that product be tested on 10000 incorrect forms and not accept any there is still a substantial chance of the failure rate being higher than 1:10000, in the same way that you can roll a die 6 times without getting a 6.

Sticking with the die in order to keep the numbers small, the chances of 10 random throws not including a 6 are:

P(0 sixes) = (⁵/₆)¹⁰ = 16%.

Now we can introduce the concept of a "confidence level" or tolerance. If the die were loaded we might theorise that this particular die had a higher than usual chance of throwing a 6: if we have 10 throws and no sixes we can be 84% confident that the probability of a six is  ¹/₆ or less.

Similarly the chances of 0 failures in 10000 attempts when the actual failure rate is1:10000 is given by:

P (0 failures) = (⁹⁹⁹⁹/₁₀₀₀₀)¹⁰⁰⁰⁰ = 37%

So we can only state with 63% confidence that the failure rate is 1:10000 or less: it could well be a little higher (for example, for a failure rate of 11:100000:

P (0 failures) = (⁹⁹⁹⁸⁹/₁₀₀₀₀₀)¹⁰⁰⁰⁰ =  33%

we have almost the same chance of seeing no failures, so you can see why we need to think carefully about our fit criterion and the associated tests.

Fortunately we don't need to become expert statisticians for this as there are two factors that simplify our calculations. Firstly we are dealing with a maximum acceptable failure rate and we are anticipating in fact that we will see no failures at all, and secondly we are dealing with sufficiently small probabilities that the chances of two or more failures should be negligible (I will demonstrate this in a subsequent post).

Our fit criterion for a statistical test should include not only the maximum acceptable failure rate but also the confidence level. So we might have:

The system shall reject at least 9999 out of every 10000 forms that are faulty according to [associated functional requirement] and this figure shall be demonstrated to a 99% confidence level.

There are mathematical ways of defining the number of tests required (I will deal with them in a subsequent post) but initially and in most cases a heuristic approach will be acceptable. Let's look for example at a test with 50000 runs and no failures:

P (0 failures) = (⁹⁹⁹⁹/₁₀₀₀₀)⁵⁰⁰⁰⁰ = 0.7%

Meaning that if 50000 tests produce 0 failures the chances of the failure rate being 1:10000 or greater are only 0.7%, or put another way:

we can be 99% confident that the failure rate is 1:10000 or less.

We are nearly there now, but there is still one important point to be taken into account: what is the source of the faulty forms we are using? Ideally these will be a random sample of actual invalid customer forms, and they should not be test forms produced by the development team (use the comments to explain why!)

The system shall reject at least 9999 out of every 10000 forms that are faulty according to [associated functional requirement] and this figure shall be demonstrated to a 99% confidence level using a random selection of faulty forms from actual customer records.

It is not necessarily easy and customers may demur at the perceived costs (for example in this case of collecting and transcribing old faulty forms that they might simply have thrown away). This makes it doubly important that you as the requirements' engineer can explain the theoretical background in simple and straightforward terms: they can cut corners (it's their money) but this will reduce the confidence in the reliability figures.

Hi to all my students and anyone else who uses this blog: I am going to start using it to pass on information on requirements' management in general and course M883 in particular that I think will interest many of you.

Various events might inspire to put up a post. Firstly, I receive many queries on matters relating to the course or to study in general, and where I feel this may be of interest to others I will post it.

Secondly I will try to take a couple of important teaching points from each TMA and post something relevant on the blog. How many points and in how much detail may depend on the amount of time I have available for it.

Thirdly I will post stuff I come across myself in my work, reading or study that could be of interest.

Feedback is, as always, appreciated. If it was useful, let me know, and if not tell me why not!

Enjoy your studies ...