Copyright (c) 2009 Rory Hewitt
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

=================
DATA QUEUE SERVER
=================



What is a NEP?



A NEP (Never Ending Program) is a program which runs continuously in a batch job, waiting for 'input'. The input might come in the form of data queue entries, signals or some other method. A NEP can be designed to 'sleep' for a defined period of time before 'waking up' to check for input, or it can be designed to continuously check for input. The Data Queue Server is an example of a NEP. On the system i, a NEP may also be known simply as a 'server' (for instance, the various TCP servers such as the Apache HTTP server) and on other platforms, a NEP may also be known as a daemon.



What is the Data Queue Server?



The Data Queue Server is a NEP which runs in its own job, waiting for entries to appear on a data queue. When entries appear on the data queue, the Data Queue Server will call a specified user-defined program, passing it the contents of the data queue entry. The System i name of the Data Queue Server program is DQSVR. Within this document, the phrase 'Data Queue Server' may also refer to the job in which the DQSVR program runs.



How is the Data Queue Server useful?



The Data Queue Server can be used to 'offload' processing from one (possibly interactive) primary job, thereby freeing up the primary job. This can be very useful when the processing might be long-running, such as printing or retrieving information from a remote system. A common use for NEP's in general is in trigger programs, whereby processing that might take more than a fraction of a second is passed to the Data Queue Server, so control can be returned immediately to the application program.



How do I use the Data Queue Server?



The Data Queue Server is controlled by the DQ command. There are three things you can do with the DQ command:

1. Start the Data Queue Server
2. End the Data Queue Server
3. Send data to the Data Queue Server

When you start the Data Queue Server, you specify the data queue it should use (which must already exist), the user-defined handler program (which will be called when the Data Queue Server receives an entry from the data queue) and the data queue key value. You can specify any character string up to 16 characters in length for the key or one of the special values *JOBNBR, *JOBUSR, *JOBNAME or *NONE (whatever value you specify must be the same length as the key which was originally defined for the data queue when it was created).

When you end the Data Queue Server, you specify the data queue and the key.

When you send data to the Data Queue Server, you specify the data queue, key and the data string itself. The DQ command allows you to pass ANY string in the Data parameter. The Data Queue Server will pass the data queue entry to the handler program you specified when you started the Data Queue Server.



Can I run multiple Data Queue Servers at once?



Yes. You could have multiple Data Queue Servers all processing the same data queue or each processing a different data queue. If you have multiple Data Queue Servers processing the same data queue, then two scenarios are possible:

1. The data queue is either unkeyed or every Data Queue Server processing the data queue uses the same key. In this case, when an entry appears on the data queue, you cannot guarantee WHICH of the Data Queue Servers will process the entry - it's simply the one which gets to the entry first

2. The data queue is keyed and each Data Queue Server uses a different key. For instance, you might have a data queue which has a key length of 3 bytes and you might run the following commands:

        DQ ACTION(*START) DTAQ(QGPL/DQ1) KEY('ABC') HANDLER(QGPL/DQHDLR1)
        DQ ACTION(*START) DTAQ(QGPL/DQ1) KEY('DEF') HANDLER(QGPL/DQHDLR2)
        DQ ACTION(*START) DTAQ(QGPL/DQ1) KEY('GHI') HANDLER(QGPL/DQHDLR3)

In this case, you would have three Data Queue Servers all waiting on the SAME data queue, but each of them is waiting only for entries with THEIR three-letter key. Subsequently, when an entry is sent to the Data Queue Server as follows:

        DQ ACTION(*SEND) DTAQ(QGPL/DQ1) KEY('DEF') DATA(<data-to-process>)

then it is handler DQHDLR2 in QGPL which will be passed the data queue entry from the Data Queue Server.



How do I create the Data Queue Server?



First you need to copy the members over to your System i. That can most easily be done using FTP, as follows:

1. Copy the DQ.ZIP file to your desktop.

2. Unzip the files into e.g. C:/Temp

3. Open an MS-DOS window and type FTP at the command prompt

4. Copy and paste each of the following commands to the FTP prompt and press Enter after each one (replacing <system-name>, <user>, <password> and <mylib> with valid values for your System i name (or IP address), user profile, password and a library on your System i):

        open <system-name>
				<user>
				<password>
				quote site namefmt 1
				lcd C:/Temp
        ascii
				put dq.cmd.txt       qsys.lib/<mylib>.lib/qcmdsrc.file/dq.mbr
				put dqtest.clle.txt  qsys.lib/<mylib>.lib/qclsrc.file/dqtest.mbr
				put dqcopy.rpgle.txt qsys.lib/<mylib>.lib/qrpglesrc.file/dqcopy.mbr
				put dqhdlr.rpgle.txt qsys.lib/<mylib>.lib/qrpglesrc.file/dqhdlr.mbr
				put dqsvr.rpgle.txt  qsys.lib/<mylib>.lib/qrpglesrc.file/dqsvr.mbr
				put dqhlp.pnlgrp.txt qsys.lib/<mylib>.lib/qpnlsrc.file/dqhlp.mbr
				quit

At this point you should have the source members on your System i in QCMDSRC, QRPGLESRC, QCLSRC and QPNLSRC. After ensuring that the member types are correct (CMD, RPGLE and PNLGRP respectively), compile the following members using the appropriate command (CRTCMD, CRTBNDRPG or CRTPNLGRP):

DQ       *CMD
DQSVR    *RPGLE
DQHDLR   *RPGLE
DQHLP    *PNLGRP

The DQCOPY RPGLE source member is used by both DQSVR and DQHDLR and can be copied into any Data Queue Server handler programs you may create. You may optionally compile the DQTEST CLLE source member using CRTBNDCL - this creates a 'test harness' showing you how to start, send data to and end a Data Queue Server. I have added 'suggested' compile defaults at the top of each source member (in the '*O:' comment line), such as DBGVIEW(*ALL) for the RPG programs.

Note that DQHDLR is an EXAMPLE Data Queue Server handler program, which is simply used to show how a handler program works. You may copy it and use it as the basis of your own handler programs, as long as your own handler programs use the same parameters. You can use the prototypes in the DQCOPY source member to call the Data Queue Server from an RPG program instead of using the DQ command - this will allow you to pass complex data in the Data field (such as a data-structure containing packed or binary fields, for instance).



What if I have more questions?



First, since you have the source, simply try it out in debug. Contact me at rory.hewitt@gmail.com if you have further queries or suggestions.





