PyDC converter (was: dragon 32 cassette format ?)

[jedie]

Now i try to handle the line numbers in the right way. But no success yet.

Test BASIC code:

Code: Select all

1 PRINT "LINE NUMBER TEST"
10 PRINT 10
100 PRINT 100
1000 PRINT 10000
32768 PRINT 32768
63999 PRINT "END";63999

This is the raw data block of the code:

0x1e,0x1a,0x0,
0x1,0x87,0x20,0x22,0x4c,0x49,0x4e,0x45,0x20,0x4e,0x55,0x4d,0x42,0x45,0x52,0x20,0x54,0x45,0x53,0x54,0x22,0x0,
0x1e,0x23,0x0,
0xa,0x87,0x20,0x31,0x30,0x0,
0x1e,0x2d,0x0,
0x64,0x87,0x20,0x31,0x30,0x30,0x0,
0x1e,0x38,
0x3,0xe8,0x87,0x20,0x31,0x30,0x30,0x30,0x0,
0x1e,0x44,
0x27,0x10,0x87,0x20,0x31,0x30,0x30,0x30,0x30,0x0,
0x1e,0x50,
0x80,0x0,0x87,0x20,0x33,0x32,0x37,0x36,0x38,0x0,
0x1e,0x62,
0xf9,0xff,0x87,0x20,0x22,0x45,0x4e,0x44,0x22,0x3b,0x36,0x33,0x39,0x39,0x39,0x0,
0x0,0x00

Normally 0x87 is PRINT...
I marked the 1Bit line numbers: 1, 10 and 100... and then?

EDIT: I make the value 10, 100, 1000 and 10000 in blue

So 0x1e,0x38,0x3,0xe8 must be the line number 1000 and 0x1e,0x44,0x27,0x10 must be 10000, isn't it?

EDIT2: And 32768 must be 0x33,0x32,0x37,0x36,0x38
and 63999 -> 0x36,0x33,0x39,0x39,0x39

EDIT3: Mark all 0x87 which must be PRINTs also.

EDIT4: Mark these line numbers:
0x3,0xe8 -> 1000
0x27,0x10 -> 10000
0x80,0x0 -> 32768
0xf9,0xff -> 63999

[jedie]

word PTR_NEXT_LINE
word LINE_NUM
byte[] TOKENISED_DATA
byte $00 End of line delimiter

'Words' are 16-bit values, two bytes, so the thing is as follows:

Code: Select all

0x1e12 Memory address of next line
0x000a Line number 10


...FOR I = 1 TO 10...

0x0 << End of line delimiter
0x1e29 memory address of the next line
0x0014 Line Number 20

...PRINT I;"HELLO WORLD!"...

0x0 << End of line delimiter
0x1e31 memory address of the next line
0x001e Line Number 30

...NEXT I...

0x0 << End of line delimiter
0x0 0x0 << end of code

Memory addresses are expressed as 16bit values, thus giving a full range of 64KB accessible by the processor.

Hm! In the case of the data from previous post it's:

0x1e,0x1a -> 0x1e1a -> 7706
0x1e,0x23 -> 0x1e23 -> 7715
0x1e,0x2d -> 0x1e2d -> 7725
0x1e,0x38 -> 0x1e38 -> 7736
0x1e,0x50 -> 0x1e50 -> 7760
0x1e,0x62 -> 0x1e62 -> 7778

What can i do with this values to get the line number and the line code?

[robcfg]

Dude, you really forgot how to decode the basic you did a couple of pages ago?

I annotated your raw data. Note that line numbers are two-byte (or Word) values.

0x1e,0x1a, // Two bytes line pointer in memory
0x0,0x1 (Line number 1),0x87(PRINT),0x20( space),0x22("),0x4c(L),0x49(I),0x4e(N),0x45(E),0x20(space),0x4e(N),0x55(U),0x4d(M),0x42(B),0x45(E),0x52(R),0x20(space),0x54(T),0x45(E),0x53(S),0x54(T),0x22("),0x0(End of line),
0x1e,0x23, // Two bytes line pointer in memory
0x0,0xa(Line number 10),0x87(PRINT),0x20(space),0x31(1),0x30(0),0x0(End of line),
0x1e,0x2d, // Two bytes line pointer in memory
0x0,0x64(Line number 100),0x87(PRINT),0x20(space),0x31(1),0x30(0),0x30(0),0x0(End of line),
0x1e,0x38, // Two bytes line pointer in memory
0x3,0xe8(Line number 1000),0x87(PRINT),0x20(space),0x31(1),0x30(0),0x30(0),0x30(0),0x0(End of line),
0x1e,0x44, // Two bytes line pointer in memory
0x27,0x10(Line number 10000),0x87(PRINT),0x20(space),0x31(1),0x30(0),0x30(0),0x30(0),0x30(0),0x0(End of line),
0x1e,0x50, // Two bytes line pointer in memory
0x80,0x0(Line number 32768),0x87(PRINT),0x20(space),0x33(3),0x32(2),0x37(7),0x36(6),0x38(8),0x0(End of line),
0x1e,0x62, // Two bytes line pointer in memory
0xf9,0xff(Line number 63999),0x87(PRINT),0x20(space),0x22("),0x45(E),0x4e(N),0x44(D),0x22("),0x3b(;),0x36(6),0x33(3),0x39(9),0x39(9),0x39(9),0x0(End of line),
0x0,0x00 (End of program)

[jedie]

Yes sometimes it's difficult

My Problem is that i what to parse it in python and i have split it by 0x00... But that's the wrong idea...

I have also "convert" the code from above in a clearer form:

Code: Select all

0x1e,0x1a,0x0,0x1,PRINT "LINE NUMBER TEST",0x0,
0x1e,0x23,0x0,0xa,PRINT 10,0x0,
0x1e,0x2d,0x0,0x64,PRINT 100,0x0,
0x1e,0x38,0x3,0xe8,PRINT 1000,0x0,
0x1e,0x44,0x27,0x10,PRINT 10000,0x0,
0x1e,0x50,0x80,0x0,PRINT 32768,0x0,
0x1e,0x62,0xf9,0xff,PRINT "END";63999,0x0,
0x0,0x0

Maybe a algorithm can look like this:

get 2Byte -> line pointer in memory
get next 2Byte -> Line number
get bytes until 0x00 -> code

right?

Seems to be easy, **now**

[jedie]

Quick and dirty:

Code: Select all

data = (0x1e,0x1a,0x0,0x1,0x87,0x20,0x22,0x4c,0x49,0x4e,0x45,0x20,0x4e,0x55,0x4d,0x42,0x45,0x52,0x20,0x54,0x45,0x53,0x54,0x22,0x0,0x1e,0x23,0x0,0xa,0x87,0x20,0x31,0x30,0x0,0x1e,0x2d,0x0,0x64,0x87,0x20,0x31,0x30,0x30,0x0,0x1e,0x38,0x3,0xe8,0x87,0x20,0x31,0x30,0x30,0x30,0x0,0x1e,0x44,0x27,0x10,0x87,0x20,0x31,0x30,0x30,0x30,0x30,0x0,0x1e,0x50,0x80,0x0,0x87,0x20,0x33,0x32,0x37,0x36,0x38,0x0,0x1e,0x62,0xf9,0xff,0x87,0x20,0x22,0x45,0x4e,0x44,0x22,0x3b,0x36,0x33,0x39,0x39,0x39,0x0,0x0,0x0)

def to16bit(values):
    return (values[0]<<8) | values[1]

def list2generator(l):
    for item in l:
        yield item

def get_from_generator(g, count):
    result = []
    for i in xrange(count):
        try:
            result.append(g.next())
        except StopIteration:
            break
    return result

def get_until(g, until):
    result = []
    while True:
        try:
            item = g.next()
        except StopIteration:
            return result
        result.append(item)
        if item == until:
            return result

def debug_with_ascii(data):
    char_list = []
    txt = ""
    for no in data:
        if no in (32,34) or (no>=48 and no<=90):
            txt += chr(no)
        else:
            if txt:
                char_list.append(txt)
                txt = ""
            char_list.append(str(hex(no)))
    print ",".join(char_list)

g = list2generator(data)
while True:
    line_pointer = get_from_generator(g, 2)
    if line_pointer == [0x00, 0x00]:
        print "---EOF---"
        break

    line_pointer = to16bit(line_pointer)
    print "line Pointer:", line_pointer

    line_number = get_from_generator(g, 2)
    line_number = to16bit(line_number)
    print "line number:", line_number

    line_code = get_until(g, 0x00)
    debug_with_ascii(line_code)

    print "---------"

output:

Code: Select all

line Pointer: 7706
line number: 1
0x87, "LINE NUMBER TEST",0x0
---------
line Pointer: 7715
line number: 10
0x87, 10,0x0
---------
line Pointer: 7725
line number: 100
0x87, 100,0x0
---------
line Pointer: 7736
line number: 1000
0x87, 1000,0x0
---------
line Pointer: 7748
line number: 10000
0x87, 10000,0x0
---------
line Pointer: 7760
line number: 32768
0x87, 32768,0x0
---------
line Pointer: 7778
line number: 63999
0x87, "END";63999,0x0
---------
---EOF---

[robcfg]

Yep, that looks nice!

You only have to translate the tokens, remove the line pointer and the end 0x00 character, and you'll have the basic listing in ASCII format.

Coding back to a .cas file may be trickier, as I don't know how to calculate the line pointers.

[jedie]

I don't know how to calculate the line pointers.

Yes, that's the question...

Is it a offset + count of all chars/tokens?

[jedie]

So, the line numbers are now ok, with: https://github.com/jedie/python-code-sn ... af69545baa

Next steps: Test checksum and ASCII Basic.

[jedie]

Note, i have moves all dragon 32 source into a separate git repro: https://github.com/jedie/PyDragon32

[jedie]

Have start to handle ASCII-Basic. But it's not complete yet: https://github.com/jedie/PyDragon32/com ... 3a1d0c3d2f

The raw data looks like this:

Code: Select all

\r1 PRINT "LINE NUMBER TEST"\r
\r10 PRINT 10\r
\r100 PRINT 100\r
\r1000 PRINT 1000\r
\r10000 PRINT 10000\r
\r32768 PRINT 32768\r
\r63999 PRINT "END";63999\r

Hat some questions:

Is it possible to mix BASIC, DATA and BINARY data into one file?
Is it possible to insert nullbytes ($00) into BASIC code? Or \r char into code? Think this can't be done or the chars would be escaped?

Maybe i will support DATA and BINARY files, too: Read somewhere that a DATA file is simple ASCII. So i can store it as a plaintext file.
BINARY: I can store it in a binary file, too.

See basically no difficulties to create a WAV from BAS,DAT,BIN files later... But:
How to store the "meta" information like "default EXEC address" and "default load address"...
What's about the "line pointer" information in tokenized BASIC? Thsi information doesn't exist in ASCII BASIC... Then can not be that important. But maybe it must be exist ? Or can just be filled with 0x0000 ? Or how to create them from a plain text BASIC code?

Maybe i will first implement only a BAS2ASCII-BASIC function...

EDIT: Next question:
What does this exactly mean:

5.4 A gap flag to indicate whether the
data stream is continuous (00) as
in binary or BASIC files, or in blocks
where the tape keeps stopping (FF) as
in data files.

EDIT2: Idea to store meta info: Add a file header in a BAS file. Maybe something like this:

Code: Select all

' created with PyDC on 24.12.2013 20:00
' origin filename: FOOBAR
' origin filetype: tokenized BASIC
' default EXEC address: 0x????
' default load address: 0x????
10 PRINT "Here comes the plain BASIC codelines"