Difference between revisions of "Tape\Disk Preservation"

From The Dragon Archive
m (Cue extension: Minor cosmetic edit)
Line 67: Line 67:
  
 
More information here: https://github.com/jedie/PyDragon32/tree/master/PyDC
 
More information here: https://github.com/jedie/PyDragon32/tree/master/PyDC
 +
 +
=== CAS -> BAS ===
 +
 +
Quick tool for converting Dragon CAS format to readable text of basic listing:
 +
 +
https://bitbucket.org/jimbro1000/cas2bas/src/master/
  
 
=== Disk to VDK ===
 
=== Disk to VDK ===

Revision as of 13:31, 28 June 2020

(Community info on how to preserve existing tape and disk software - conversion from native Dragon to PC formats - WAV, CAS, VDK, DMK, conversion between formats and restoration back to physical media)

Tools

WAV/VOC to CAS

XROAR

Currently the preferred method is the XROAR emulator, this can be using to output the byte stream that its virtual ADC reads from a WAV file as the tape loads (CLOAD/CLOADM). Because this method is using an emulated ADC and the actual Dragon ROM routines to read the "tape" the success ratio for conversion is much higher.

Dragon Convert for Windows (DCWIN)

DCWIN is a Windows application that can perform conversions of 8-bit WAV files to CAS format for use with emulators. Its method and style of operation is broadly similar to DC (see below) together with additional options and enhancements for preserving and restoring leaders. It has a graphical user interface and requires the .NET framework version 3.5 (which is supplied with Windows 7 and can also be installed on Windows XP, Windows Vista and Windows Server 2003/2008).

Dragon Convert (DC)

Dragon Convert (DC) is the original tool for converting Dragon and Coco cassette data between a range of different file formats. It's designed for MS-DOS and has always been packaged with the PC-Dragon emulator series. The following notes are taken from the supplied Reference Manual...

Where the input to Dragon Convert is a WAV or VOC file the audio information is scanned for Dragon cassette data and converted to a virtual cassette file. The output filename is derived from the Dragon filename rather than the input filename.

During conversion Dragon Convert displays a counter to show the progress in converting the files. If the cassette programs conform to the conventional Dragon cassette format then the filenames and load/execution addresses of binary files are listed as the programs are encountered. All of the files contained in a single input file will be placed in a single output file.

If Dragon Convert detects a loading error in the WAV or VOC file then you are given the option to abort the conversion, retry a block, or proceed with the conversion ignoring the error. If you choose to ignore errors then the converted file is unlikely to be usable, although several programs do exist which deliberately contain errors.

If you choose to retry a block then the current conversion parameters (see below) appear so that different parameters may be entered. Type the new values at the prompt or press RETURN to leave a parameter unchanged. Dragon Convert then attempts to load the block using the new parameter values. There is, of course, no advantage to be gained by retrying a block without adjusting either of the parameters.

Dragon Convert will normally assume that the sampled file begins with a leader, and will initially seek the leader to synchronise with the data. In cases where a leader is not present this may cause data to be missed. To avoid this give the /L option on the command line to disable the feature.

Although Dragon Convert is designed to work with data in the conventional Dragon cassette format it may also be used with cassettes which use a proprietary format. For such programs specify /R on the command line and Dragon Convert will scan the input as a raw bit stream. This results in a file called CASSETTE.CAS. When Dragon Convert is used with this option the display shows only the number of bytes converted and loading errors cannot be reported.

The following actions may be taken if you suffer from loading errors:

  • Use the highest available sampling rate for your sound card. At least 22KHz is required.
  • The amplitude of the input data should comfortably span the available range. Increase the input volume if necessary.
  • It is a good idea to clean up the wave form before presenting it for conversion. Try cutting any white space before the first program block and omitting any noisy parts of the leaders.
  • MS-Windows software (Eg. Creative Wave Studio) usually produces better results than MS-DOS based software.
  • The wavelength amplitude threshold may be specified on the command line using the /Wn option. For a 44KHz sample a threshold of 29 to 31 is usually correct. For 22KHz files the value of n should be in the range 12 to 15.
  • The relative volume of the wave form may be specified using the /Sn command line option, where 0 < n < 126.

You can abort the operation of Dragon Convert in the middle of a conversion by pressing the CTRL key.

CAS to WAV

Dragon Convert for Windows (DCWIN)

DCWIN can be used to convert cassette files to WAV format in a similar way to DC (see below). The size of file that can be converted is limited only by the available memory to buffer the WAV file (unlike the MS-DOS version which cannot be used for CAS files of 64KB or greater).

Dragon Convert (DC)

A secondary use of Dragon Convert (DC.EXE) is to convert cassette files back to digital audio format so that they can be written to cassette and loaded into a real machine. When converting to audio format the cassette file is not examined thereby enabling any file type or proprietary format to be converted. To use Dragon Convert for this purpose simply specify the filename of a cassette file when you start the program.

By default a 44KHz WAV file will result. This gives very high quality playback but requires more than 8Mb of hard disk space for a 32K Dragon file. You can therefore optionally specify /22 on the command line to request a 22KHz output file (or /11 for 11KHz).

If the CAS file has sufficient SYNC bytes (see CAS Format below) then the resultant WAV can be loaded by a real Dragon 32/64. Some of the older CAS files with truncated SYNC bytes can be rescued using the FIXCAS.EXE tool that attempts to insert the appropriate number of missing SYNC bytes into the CAS file.

BAS <-> CAS <-> WAV

PyDC - Python Dragon 32 converter

PyDC is a OpenSource (GPL) Python script to convert between .bas <-> .cas <-> .wav in every variation.

More information here: https://github.com/jedie/PyDragon32/tree/master/PyDC

CAS -> BAS

Quick tool for converting Dragon CAS format to readable text of basic listing:

https://bitbucket.org/jimbro1000/cas2bas/src/master/

Disk to VDK

Using and old PC with a 5.25 inch drive and a DOS environment, the VCOPY.EXE tool can be used to copy some common Dragon disk formats to .VDK files.

Making DragonDos disk images under Linux

This will allow you to make RAW images of DragonDos disks, you will need to add/remove the VDK header to allow them to be used with an emulator. For most of this you will need to be logged in as root. Firstly you need to add the following lines to the end of your /etc/fdprm

DragonDos 40 track disk in 80 track drive (eg 1.2M). for 40 track in 40 track drive, set the 6th column to 0 instead of 1 (like the 80 track formats below !).

ddosSS40 360 9 1 40 1 0x2A 0x39 0xDF 0x50

ddosDS40 720 9 2 40 1 0x2A 0x39 0xDF 0x50


DragonDos 80 track disks in an 80 track drive (eg 720K,1.2M,1.44M)

ddosSS80 720 9 1 80 0 0x2A 0x39 0xDF 0x50

ddosDS80 1440 9 2 80 0 0x2A 0x39 0xDF 0x50

Then change to the directory that you want to create the image in and type :- setfdprm /dev/fd0 ddosSS40 Replace /dev/fd0 with the name of the floppy drive you are using, and ddosSS40 with the name of the format you want to use in this case a single sided 40 track disk.

IMPORTANT

The parameters set by setfdprm must be set AFTER the disk has been inserted into the drive, and will need to be set FOR EACH subsiquent disk, don't say you have not been warned :)

To actually create the image type :- dd if=/dev/fd0 count=360 of=dragon.img Replace /dev/fd0 with the name of the floppy you are using, and dragon.img with the name of the file you want to create. Count, should be replaced with the first number after the disk type (second column), from /etc/fdprm, this is the size of the disk in 512 byte blocks.

You can also write a raw disk image to a preformatted DragonDos disk with :- dd if=dragon.img count=360 of=/dev/fd0 Parameters as described above in reading the disks.

You will then need to add/remove the VDK header - a set of tools is available to help.


Disk to JVC

Linux - dd


JVC to VDK

Perl Tool to add VDK header


VDK to JVC

Perl Tool to remove VDK header

VDK to Disk

Using and old PC with a 5.25 inch drive and a DOS environment, the VCOPY.EXE tool can be used to copy some common Dragon disk formats from .VDK files to disks. The disks must have been formatted in a real Dragon first....

JVC to Disk

Linux - dd

Dumping ROMS

The PC-Dragon Reference Manual provides the following advice for dumping ROMs...

There are several ways to transfer your ROM for use with Dragon emulators. The method suggested here is to use the Dragon Convert utility to transfer the ROM in the same way as cassettes. An alternative method would be to place the ROM in an EEPROM reader connected to a PC and dump the contents to a file.

The procedure for transfer via cassette and the Dragon Convert utility is as follows:

    1)  Boot up your Dragon 32, Dragon 64 or Tandy CoCo.
         
    2)  Ready  your  tape  recorder for recording  and  commit  the  ROM  to
         cassette by typing:
         
                     CSAVEM "DRAGROM", &H8000, &HBFFF, 0
                                      
        If  you  have a Tandy CoCo you may optionally use the name  TANDYROM
         rather than DRAGROM.
         
    3)  Boot  up  your  PC  and SoundBlaster Pro wave form editor  software.
         Sample  the cassette recorded in step 2 into a file. Alternatively,
         use  the  parallel cable and the ReadVoc utility.  A  large  output
         file should result.
         
    4)  Now use Dragon Convert to extract the cassette data. Type:
         
                                 DC  DRAGROM
                                      
        Please  don't  expect the conversion to work on the  first  attempt.
         You  are  likely to need to adjust the command line switches  until
         you  find  a combination which is suited to your hardware.  If  you
         still  have no success then you may wish to try repeating  step  3.
         Upon successful conversion the file DRAGROM.CAS will be created.
         
    5)  Now  use  Dragon  Convert again to convert the cassette  data  to  a
         cartridge file. Type:
         
                             DC  /D  DRAGROM.CAS
                                      
        This should produce the file DRAGROM.DGN in the current directory.
         
    6)  If  you  have  a  Dragon  64 then you may optionally  also  wish  to
         transfer  the 64K version of the Dragon ROM. This will  enable  you
         to  use  the emulator in 64K BASIC mode. The procedure to  transfer
         this  second  ROM  is much as before. Boot up your  Dragon  64  and
         switch  to  64K mode by typing `EXEC 48000'. Now commit the  second
         ROM to cassette by typing:
         
                     CSAVEM "D64ROM2", &HC000, &HFEFF, 0
                                      
        Transfer  this  file  to  your PC by using the  procedure  described
         before  in  steps 3 to 5. This time the result should be  the  file
         D64ROM2.DGN.
         

File Formats

CAS File Format

The CAS file format was originally created for the PC-Dragon emulator and is one way of representing programs or data stored on Dragon cassette tapes. The CAS file format comes in two flavours, the most usual and default format is modelled after how the Dragon actually stores data on cassettes, optionally with truncated leader bytes to minimise the file size. The alternate format is simply the RAW bit stream and was only used for games that did not conform to the standard tape format (an early form of copy protection).

The standard Dragon tape format is:

1. A leader block of $55 multiplied by the 16 bit number in location $90:91 (default 128). 2. A namefile block. 3. A blank section of tape for processing of the namefile block. 4. Another leader block of $90:91 bytes of $55 5. One or more data blocks. 6. An end-of-file block.

A header block, data block or EOF file block consists of:

1. A leader byte - $55 2. A sync byte - $3C 3. A block type byte:

                     00=namefile block
                     01=data block
                     FF=end-of-file block

4. A block length byte (0-255)

5. 0-255 bytes of data. For a namefile block this consists of:

    5.1 An 8 byte program name
    5.2 A file ID byte where:
        00=BASIC program
        01=Data file
        02=Binary file
    5.3 An ASCII flag where:
        00=Binary file
        FF=ASCII file
    5.4 A gap flag to indicate whether the
        data stream is continuous (00) as
        in binary or BASIC files, or in blocks
        where the tape keeps stopping (FF) as
        in data files.
    5.5 Two bytes for the default EXEC address
        of a binary file.
    5.6 Two bytes for the default load address
        of a binary file.

For a data block, this consists of the actual data to load/save and there is no data associated with an EOF block.

6. A checksum byte which is: sum of all the data bytes + block type + block length.

7. A trailer byte - $55


The first byte of a CAS file must be $55 for identification purposes. The first block of a standard format CAS file should be a namefile block, and the last block is usually an EOF block. Some games used copy protection where the number of blocks to be loaded did not match the number specified in the file header and a fake EOF block was included so that a simple copy of the file would result in a truncated file (other similar mechanisms skipped blocks or used non-standard block types).

Note that standard format CAS files may omit leaders. If leaders are required, use the FIXCAS utility to add leaders to more closely resemble the true Dragon cassette tape format. Since the standard format CAS file is a fairly simple representation of the data stream read from a tape after demodulation and decoding of the audio signals, certain properties of cassette tapes cannot be reproduced, e.g. gaps of silence (at least one game loader uses this as a copy protection mechanism that needs the RAW format of CAS to work - although this doesn't store silence it does store noise in those gaps).

The above description makes it pretty easy to create CAS files. Reading CAS files can be a bit more difficult though. Data is actually represented as a stream of bits on cassette tapes, and the Dragon uses different audio signals to represent different bit values. When decoding such an audio signal into a bit stream and creating a CAS file, the structure described above might not align to byte boundaries in the CAS file (this can occur when using DC, although DCWIN does synchronise bitstream conversions to byte boundaries). Especially the leader may not be decoded to an integral number of bytes. To make matters even worse, both leaders and sync bytes may contain noise and still be readable. A robust CAS file reader must take this into account.

For an example of a RAW format CAS file exhibiting both byte misalignment and leader noise, see Beyond Software's Kriegspiel (last modified 2009-01-04) in the download area.

Cue extension

This is an extension to the format, created by robcfg and polished and implemented on Xroar by sixxie.

The idea is to add missing information about wave frequencies and silence between blocks to preserve the original structure of the tape and support custom loaders and protection schemes that rely on different frequencies or periods of silence in strategic places.

It also allows exporting the cas file as wav with proper frequencies and silences.

The extension adds a chunk of data at the end of a standard cas file so that it's completely backwards compatible.

The format of the extra data is as follows:

1. Cue data start marker (4 bytes). Value is '[CUE' in ascii and 0x5B435545 in hex.

2. Cue data blocks. Cue data is stored in blocks with a standard format:

    Type (1 byte)
    Size (variable-length uint31, see below for explanation)
    Data

Currently there are 3 types of blocks, with the possibility of adding more.

    0xF0 Timing (4 bytes). Contains two uint16 values. The frequencies in Hz of the 0 bit and the 1 bit.
    0x00 Silence (2 bytes). Contains silence duration in ms as an uint16.
    0x0D Data (8 bytes). Start and end offsets of a block of data on the cas file as two uint32 values.

3. Cue data start offset. An uint32, which is the offset on the cas file of the cue data start marker.

4. Cue data end marker ('CUE]' in ascii and 0x4355455D in hex).

This is done to make easy to detect the extended cas files by checking the last 4 bytes of the file for the 'CUE]' marker and if found, read the offset of the cue data start.

The block size is stored as variable-length uint31, which is defined as follows:

    * 7-bit        0nnnnnnn
    * 14-bit       10nnnnnn nnnnnnnn
    * 21-bit       110nnnnn nnnnnnnn nnnnnnnn
    * 28-bit       1110nnnn nnnnnnnn nnnnnnnn nnnnnnnn
    * 31-bit       11110XXX Xnnnnnnn nnnnnnnn nnnnnnnn nnnnnnnn

Leading 1 bits in initial byte determine number of additional bytes required.

Example files and original discussion of the format can be found on this thread.

JVC/DSK File Format

A disk format created by Jeff Vavasour for his Tandy emulator series, this is the simplest disk image format. It consists of an optional header followed by the DATA portion of each disk sector in order of track, then side, then sector number.

Taken from Tim Lindner's JVC format documentation, the header is as follows:

Byte Offset Description Default
0 Sectors per track 18
1 Side count 1
2 Sector size code 1
3 First sector ID 1
4 Sector attribute flag 0

Sector size code indicates sector is (128 << n) bytes long. The default of 1 means a sector size of 256 bytes.

The sector attribute flag indicates that each sector is preceeded by an attribute byte. This contains which bits would be set in the WD279x status field after a read sector command, and can indicate record type, record not found and CRC error.

The presence of the sector attribute flag confuses things slightly, as if it is set, suddenly the header size is the file size modulo 257 instead of modulo 256.

VDK File Format

The VDK file format was introduced for the PC-Dragon emulator (v2.05) as an evolution of original work by Stewart Orchard. Similar to the JVC/DSK format, this format contains a header followed by a raw dump of sector data. Data is in track, then side, then sector number order.

Complete header information from the source code:

Byte Offset Description
0, 1 'd', 'k'
2, 3 Header size (little-endian)
4 Version of VDK format
5 Backwards compatibility version
6 Identity of file source
7 Version of file source
8 Number of tracks
9 Number of sides
10 Flags
11 Compression flags and name length

DMK File Format

A file format created by David Keil, carrying a lot more information about the underlying structure of the disk. Almost every piece of information that can be reported to the WD279x is recorded.

DMK Format Documentation