POSIX Realtime Test Suite (for former 1003.1b, now the Realtime option in 1003.1-200x)
Also covers former 1003.1d, 1003.1j and 1003.1q standards - Advanced Realtime, Realtime Extensions and Tracing.
The industry standard test suite with test coverage for the IEEE
POSIX® Realtime extension. Verifies the POSIX 1003.1b-1993/1003.1i-1995 Realtime extension,
the UNIX 98 _XOPEN_REALTIME Feature Group,
POSIX 1003.1, 2003 Edition
,
the UNIX 03 _XOPEN_REALTIME and _XOPEN_ADVANCED_REALTIME Feature Groups. In the Northwood version additionally verifies the POSIX 1003.1, 2013 Edition
the UNIX V7 _XOPEN_REALTIME and _XOPEN_ADVANCED_REALTIME Feature Groups. Built upon the proven VSXgen harness, yielding
a configurable and productive tool. Using the VSXgen architecture, this suite is provided as a VSXgen
testset. Coverage for functions and header file tests. Test modes for UNIX 98, POSIX Realtime (1993), UNIX 03 and POSIX.1, 2003 edition
In the Northwood version Test modes for UNIX V7 and POSIX.1, 2013 edition
Assertions following the IEEE Std 1003.3 have been developed for
all tests, greatly aiding failure analysis. Optional for testing the Single UNIX® Specification Version 2, 3, and 4.
-
The VSRT-PSE derivative provides an embedded test capability
-
Developed, owned and supported by The Open Group.
If your organization is developing an implementation conforming to the
IEEE POSIX Realtime standards then this test suite gives significant test
coverage and is recommended for in-house quality-assurance as well as
part of your branding campaign.
Keywords: POSIX® Realtime, 1003.1b, UNIX® 98, UNI® 03, POSIX 1003.1,2003 Edition, UNIX® V7, POSIX 1003.1, 2013 Edition,
Realtime, Testing, POSIX, X/Open.
Top
The Open Group's Realtime Test Suite, known as VSRT, is a stand alone test suite
covering POSIX 1003.1b, 1003.1d, 1003.1j, 1003.1q and their associated Technical Corrigenda.
It includes test modes for UNIX 03,
IEEE Std 1003.1, 2003 Edition, UNIX V7 and IEEE Std 1003.1, 2013 Edition
VSRT uses the robust VSXgen test harness, which in turn uses the industry
standard Test Environment Toolkit (TET3) . The benefits of using VSXgen,
which is derived from VSX4, is that the harness is familiar to existing
users of The Open Group's Test suites, it has proven portability and robustness
characteristics and it allows the VSRT tests to be integrated with other
testsets that run under VSXgen, making the test suites easily useable
for in-house quality assurance and automated regresssion testing.
TET3 is the latest version of the Test Environment Toolkit and supports
multi-threaded testing. More information on the Test Environment Toolkit
is described on the TETworks web
site.
Top
The test suite covers the following functional areas in the IEEE POSIX
1003.1b/1003.1d/1003.1j/1003.1q standards, IEEE Std 1003.1, 2003 Edition/ IEEE Std 1003.1, 2013 Edition/ UNIX 98/ UNIX 03/UNIX V7.
POSIX 1003.1b Test Extension
- Realtime Signals
Tested by generating series of realtime signals to verify queueing,
reliability, prioritization, and differentiation between mutiple occurrences.
- Synchronized input and output
Tested by generating I/O and verifying it is appropriately synchronized
on request. As synchronization can occur independent of this API,
this testing will be less than completely comprehensive.
- Asynchronized input and output
Tested by generating series of asynchronous I/O requests and verifying
queueing, logical parallelism, signal generation, event cancellation,
event waiting, etc.
- Semaphores
Tested by generating multiple processes and verifying they can correctly
synchronize operation with semaphores under various conditions.
- Memory locking
Tested by locking and unlocking process memory. As the effects of
locking are difficult to discern, this testing will be less than comprehensive.
- Memory mapped files and shared memory
Tested by creating and manipulating multiple shared memory objects
and mapped files.
- Priority scheduling
Tested using multiple processes to verify SCHED_FIFO and SCHED_RR
scheduling algorithms operate as specified. SCHED_OTHER will also
be verified if identical to SCHED_FIFO or SCHED_RR. The timing and
ordering of process scheduling will be verified for the defined algorithms
and parameters.
- High resolution clocks and timers
Tested by creating and manipulating various timers to verify granularity,
accuracy, signal generation, etc.
- IPC Message passing
Tested by generating multiple processes and verifying they can correctly
communicate messages under various conditions.
The POSIX Realtime functions tested are:
aio_cancel() aio_error() aio_fsync() aio_read()
aio_return() aio_suspend() aio_write()
clock_getres() clock_gettime() clock_settime()
fdatasync()
lio_listio()
fchmod()
mlock() mlockall()
mmap() munmap() mprotect()
mq_close() mq_getattr() mq_notify() mq_open()
mq_receive() mq_send() mq_setattr() mq_unlink()
ftruncate()
msync() fsync()
munlock() munlockall()
nanosleep()
sched_get_priority_max() sched_get_priority_min() sched_getparam()
sched_getscheduler() sched_rr_get_interval() sched_setparam()
sched_setscheduler() sched_yield()
sem_close() sem_destroy() sem_getvalue() sem_init()
sem_open() sem_post() sem_trywait() sem_unlink() sem_wait()
shm_open() shm_unlink()
sigqueue() sigtimedwait() sigwaitinfo()
timer_create() timer_delete() timer_getoverrun() timer_gettime()
timer_settime()
The headers to be tested are:
<aio.h> <mqueue.h> <sched.h> <sys/mman.h>
POSIX 1003.1j Test Extension
The test suite covers the following
functional areas in the IEEE POSIX 1003.1j standard.
Memory management
A facility to allow programs to allocate or access different kinds of physical memory that are present in the
system, and allow separate application programs to share portions of this memory.
Synchronization
Synchronization primitives that allow multiprocessor applications to achieve the
performance benefits of their hardware architecture.
Clocks and Timers
The addition of the Monotonic Clock, the specification of the effects of setting the time of a clock on other timing
services, and the addition of functions to support relative or absolute suspension based upon a clock specified by the application.
The following new POSIX functions are
tested:
posix_typed_mem_open()
posix_mem_offset ()
posix_typed_mem_get_info()
pthread_spin_init()
pthread_spin_destroy()
pthread_spin_lock()
pthread_spin_trylock()
pthread_spin_unlock()
pthread_barrierattr_init()
pthread_barrierattr_destroy()
pthread_barrierattr_getpshared()
pthread_barrierattr_setpshared()
pthread_barrier_init()
pthread_barrier_destroy()
pthread_barrier_wait()
pthread_rwlockattr_init()
pthread_rwlockattr_destroy()
pthread_rwlockattr_getpshared()
pthread_rwlockattr_setpshared()
pthread_rwlock_init()
pthread_rwlock_destroy()
pthread_rwlock_rdlock()
pthread_rwlock_tryrdlock()
pthread_rwlock_timedrdlock()
pthread_rwlock_wrlock()
pthread_rwlock_trywrlock()
pthread_rwlock_timedwrlock()
pthread_rwlock_unlock()
clock_nanosleep()
pthread_condattr_setclock()
pthread_condattr_getclock()
Tests are also performed on existing
functions POSIX .1 functions whose behaviour is modified by the
advanced realtime functionality.
The following header files are -
tested:
<sys/mman.h>
<pthread.h>
<stat.h>
<time.h>
<sys/types.h>
<unistd.h>
POSIX 1003.1d Test Extension
The test suite covers the following
functional areas in the IEEE POSIX 1003.1d standard.
Spawn a Process
New system services to spawn the
execution of a new process in an efficient manner.
Timeouts for some blocking services
Additional services that provide a
timeout capability to system services already defined in POSIX.1b and
POSIX.1c, thus allowing the application to include better error
detection and recovery capabilities.
Sporadic Server Scheduling
The addition of a new scheduling policy
appropriate for scheduling aperiodic processes or threads in hard
real-time applications.
Execution Time Clocks and Timers
The addition of new clocks that measure
the execution times of processes or threads, and the possibility to
create timers based upon these clocks, for runtime detection (and
treatment) of execution time overruns.
Advisory Information for File Management
Addition of
services that allow the application to specify advisory information
that can be used by the system to achieve better or even
deterministic response times in file management or input & output
operations.
The following new POSIX functions are tested:
posix_spawn()
posix_spawn_file_actions_addclose()
posix_spawn_file_actions_adddup2()
posix_spawn_file_actions_addopen()
posix_spawn_file_actions_destroy()
posix_spawn_file_actions_init()
posix_spawnattr_destroy()
posix_spawnattr_getflags()
posix_spawnattr_getpgroup()
posix_spawnattr_getschedparam()
posix_spawnattr_getschedpolicy()
posix_spawnattr_getsigdefault()
posix_spawnattr_getsigmask()
posix_spawnattr_init ()
posix_spawnattr_setflags()
posix_spawnattr_setpgroup()
posix_spawnattr_setschedparam()
posix_spawnattr_setschedpolicy()
posix_spawnattr_setsigdefault()
posix_spawnattr_setsigmask()
posix_spawnp()
pthread_mutex_timedlock()
sem_timedwait()
mq_timedreceive()
mq_timedsend()
clock_getcpuclockid()
pthread_getcpuclockid()
posix_fadvise()
posix_fallocate()
posix_madvise()
posix_memalign()
Tests are also performed on existing
functions POSIX .1 functions whose behaviour is modified by the
additional realtime extensions.
The following header files are tested:
<fcntl.h>
<limits.h>
<sys/mman.h>
<mqueue.h>
<pthread.h>
<sched.h>
<semaphore.h>
<spawn.h>
<stdlib.h>
<time.h>
<unistd.h>
POSIX 1003.1q Test Extension
The test suite covers the following
functional areas in the IEEE POSIX 1003.1q standard.
The following new POSIX functions are
tested:
pthread_attr_gettracingstate()
pthread_attr_settracingstate()
pthread_gettracingstate()
pthread_settracingstate()
posix_trace_attr_destroy()
posix_trace_attr_getclockres()
posix_trace_attr_getcreatetime()
posix_trace_attr_getgenversion()
posix_trace_attr_getinherited()
posix_trace_attr_getlogfullpolicy()
posix_trace_attr_getmaxdatasize()
posix_trace_attr_getlogsize()
posix_trace_attr_getstreamsize()
posix_trace_attr_getmaxsystemeventsize()
posix_trace_attr_getname()
posix_trace_attr_getstreamfullpolicy()
posix_trace_attr_getusereventsize()
posix_trace_attr_getutsname()
posix_trace_attr_init()
posix_trace_attr_setinherited()
posix_trace_attr_setlogfullpolicy()
posix_trace_attr_setmaxdatasize()
posix_trace_attr_setlogsize()
posix_trace_attr_setstreamsize()
posix_trace_attr_setname()
posix_trace_attr_setstreamfullpolicy()
posix_trace_close()
posix_trace_create()
posix_trace_create_withlog()
posix_trace_eventid_open()
posix_trace_eventid_equal()
posix_trace_trid_eventid_open()
posix_trace_eventid_get_name()
posix_trace_eventtypelist_getnext_id()
posix_trace_eventtypelist_rewind()
posix_trace_eventset_add()
posix_trace_eventset_del()
posix_trace_eventset_empty()
posix_trace_eventset_fill()
posix_trace_eventset_complement()
posix_trace_eventset_ismember()
posix_trace_flush()
posix_trace_get_attr()
posix_trace_get_filter()
posix_trace_get_status()
posix_trace_getnext_event()
posix_trace_open()
posix_trace_rewind()
posix_trace_set_filter()
posix_trace_shutdown()
posix_trace_start()
posix_trace_stop()
posix_trace_clear()
posix_trace_timedgetnext_event()
posix_trace_trygetnext_event()
posix_trace_event()
Tests are also performed on existing
functions POSIX .1 functions whose behaviour is modified by the trace
functionality.
The following header files are tested:
<pthread.h>
<trace.h>
Top
VSRT is modular, both at the implementation level and at the user level.
The testing process is divided into clear stages -
Each stage is clearly documented in the User and Installation Guide, which
accompanies the product.
This stage includes setting up user-ids and groups for testing, and
reading the tape. This stage includes configuring and installing the TET
harness, which is used to control building, execution and cleanup of VSRT
tests.
VSRT automatically installs on most common platforms. Where system dependencies
may vary, these are isolated into clearly defined user changeable sections
of the test suite. During this stage VSRT builds the test harness and
associated libraries.
VSRT separates the building of the test harness and tools from the testsets
themselves. The test suite may be built completely or in sub-sections, which
decrease in granularity from test sections and areas down to individual
testsets.
Once built, the test suite may be run as a whole in one pass, or in sub-sections
which decrease in granularity from test areas down to individual tests within
testsets. If you are producing a branding application the test suite
must run in a single uninterrupted pass.
VSRT generates an intermediate level journal file in a TET compliant
format. This is sufficient for debugging purposes. When a formal report
is required, such as for The Open Brand then the formal report writers
provided with VSRT are used to process the journal file.
Top
POSIX is a registered trademark of the IEEE
Top
|
