Real-time testing

Key takeaways

Real-time testing belongs to computer science; place it in that map before memorising details.
Learn the definition first, then one example that makes the definition concrete.
Connect Real-time testing to a quantity you can measure, compute or draw — that is where exam questions come from.
Reproduce the core statement of Real-time testing from memory before moving on to harder problems.

Reference excerpt

Real-time testing is the process of testing real-time computer systems. Software testing is performed to detect and help correct bugs (errors) in computer software. Testing involves ensuring not only that the software is error-free but that it provides the required functionality to the user. Static and conventional methods of testing can detect bugs, but such techniques may not ensure correct results in real time software systems. Real-time software systems have strict timing constraints and have a deterministic behavior. These systems have to schedule their tasks such that the timing constraints imposed on them are met. Conventional static way of analysis is not adequate to deal with such timing constraints, hence additional real-time testing is important.

Strategy Test case design for real time testing can be proposed in four steps

Task testing In the very first step, each task is tested individually with conventional static testing. This testing is performed only to discover the errors in logic or syntax of the program. Order of the events doesn't matter as task testing doesn't deal with timing constraints and time properties of events. Behavioral testing Using the system models designed with the help of automated testing tools, it is possible to simulate behavior of real time system and impact of concurrent external events on its behavior. Intertask testing Once the testing with the individual task is done, then task is supposed to be error free in coding and behavioral area. Time-related constraints are tested with intertask testing. To reveal the errors in communication, asynchronous tasks are tested with variable data rates and different payloads. System testing In this testing, software and hardware are integrated and full range of system tests are conducted to discover errors, if any, during software and hardware interfacing.

Tools for real time testing As testing of real time systems is becoming more important, there are some tools designed for such testing.

MSC Message Sequence Charts is an internationally accepted standard for capturing requirements. MSC provides a graphical 2-D language often required for collecting requirements through some interaction scenarios.

SDL Specification and Description Language is a standard used for design and analysis. SDL supports the specification of complex software systems and has been extensively applied across a broad array of domains from telecommunications, automation, through to general software development

TTCN Testing and Test Control Notation is the only internationally standard testing language. TTCN3 provides a broader applicability, as compared to earlier versions of TTCN, which were primarily focused on OSI protocols only. These three standards together are used for testing of real time applications. It is necessary that requirements be satisfied with these models and test cases generated must capture the functional and real time information needed to test systems. Also, the changes in the requirements of design and new information about the real time properties of systems should be fed into models so that its impact can be found out. To accurately capture the real time properties of a given test system and to ensure that requirements and models are used to generate realistic and enforceable timing information, it is essential that the language itself (TTCN-3) has a well understood and semantically sound model of time.

TTCN-3

Background TTCN-3 is the only currently available, internationally standardized testing language. Prior to TTCN3, its earlier versions were having limited functionality and limited scope over OSI protocol. But, TTCN3 is an advanced version and has broader applicability. Characteristics of TTCN3 are:

ability to specify dynamic concurrent testing operations for message-based and procedure based communication the ability to specify data and signature templates with powerful matching mechanisms type and value parameterization the assignment and handling of test verdicts test suite parameterization and test case selection mechanisms The reason for using TTCN3 for real time testing is because of its timers. These timers are defined in function test suites. There are no any global kind timers used in TTCN3. These timers can be started, stopped and checked using simple functions like timer.start, timer.stop, and timer.read. Snapshot Semantics is a technique in TTCN3 (also in TTCN2), which deals with the message passed during communication by system to system or implementation under test. When a series of responses are received by system under test, then snapshot is taken and they are evaluated in order of their arrival. So, each time around a set of attributes, a snapshot is taken and only those events are evaluated which are present in snapshot. But this technique is not efficient as some events and their attribute information might get lost while the snapshot is taken. Some events might get recorded on processing queue, but not on snapshot. Such events can never get processed. Also, if the test executer equipment is not fast enough, then it can not communicate properly with the system under test. So, faults might get generated during such test evaluation.

Linux tools Sources:

rt-tests: https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git/ cycletest by SUSE: https://documentation.suse.com/sle-rt/15-SP1/html/SLE-RT-all/art-slert-hwtest.html

See also Real-time Linux Embedded Linux Real-time operating system Safety-critical systems

References

Worked examples

Example 1 — a first encounter with Real-time testing

Start with the simplest possible case. Write down what Real-time testing claims or describes in one sentence, then invent the smallest concrete situation in which that sentence is true. In computer science, the smallest case is usually a single object, a single equation or a single measurement. Check that every symbol or term in your sentence has a meaning in that case.

Example 2 — changing one variable

Take the situation from Example 1 and change exactly one quantity: double it, halve it, or set it to zero. Predict what should happen to Real-time testing before you calculate. Comparing your prediction with the result is the fastest way to find out whether you understand the idea or only the words.

Example 3 — an exam-style question

Typical questions about Real-time testing ask you to (a) state it precisely, (b) apply it to given data, and (c) explain a limitation. Practise writing all three answers in under five minutes; the third part is what separates a full-mark answer from an average one.

Applications of Real-time testing

In research: Real-time testing appears in computer science research whenever the underlying quantities have to be modelled precisely. Papers usually cite it as a starting assumption and then explore where it breaks down.
In technology and industry: Engineering practice reuses Real-time testing in design rules, simulations and safety margins. Knowing the idea lets you read a specification sheet and understand why the numbers look the way they do.
In the classroom: Real-time testing is common in secondary-school and first-year university syllabi. It links to neighbouring topics Real-time computing, Software testing, so understanding it makes those chapters shorter.
In everyday life: Look for Real-time testing outside the textbook — in sport, cooking, traffic, electronics or the sky above you. An example you found yourself is remembered far longer than one you were given.

How to study Real-time testing in 20 minutes

Read the reference excerpt below once, without taking notes.
Close the page and write down what Real-time testing means in your own words.
Compare your version with the excerpt and mark what you missed.
Work through the three examples above with pen and paper.
Explain Real-time testing out loud to somebody else — or to Teacher Smith in the lgStudy chat.

Frequently asked questions

What is Real-time testing in simple terms?

Real-time testing is the process of testing real-time computer systems. Software testing is performed to detect and help correct bugs (errors) in computer software.

Why does Real-time testing matter?

Because it connects several computer science ideas at once: it gives you a definition you can apply, a quantity you can calculate, and a way to check whether a result is plausible.

How should I study Real-time testing?

Read the excerpt, restate it from memory, then work through the examples and applications listed on this page. The five-step study plan above takes about twenty minutes.

What does this page cover?

It gives you a compact reference excerpt plus original lgStudy explanations, examples, applications and study material on Real-time testing.