Firmware application version #3 release
-------------------------------------------
N64 updates:
Some extra timing delays were necessary for consistent dumping.
Includes fixes for N64 dumping that proved working for me on 10+ carts I
dumped successfully. There is now a .csv in docs folder with cart size
in bytes, divide that number by 131,072 to get size in mbits for -z
flag. Includes CRC32 checksum that can be verified using HxD hex
editor. N64 file output is big-endian so it reads as it should in hex
editor which is .z64 file format. lua script updated to print out the
game name from the in rom header.
NES updates:
flashing support for quite a few different mappers including GTROM,
MMC2, MMC4, action53, easyNSF, and other tweaks to rd/wr timings.
FIRMWARE UPDATES:
inlretro2.lua script now tries to poll the firmware "application
version" and suggest updating your firmware if you're not running the
version in this release. You can always update your firmware using the
bootloader method in the readme. But there is an easier method if you
have an stm32 based device (translated: you purchased your device in
2018 or later). AND you're already running firmware version v2.3.x
simply run the new scripts which were added in this commit:
PCB version 2.0 or 2.1 (large square PCB with option for 6 connectors)
>inlretro.exe -s scripts\inlretro_inl6fwupdate.lua
PCB version 2.0N (smaller PCB with NES connector only:
>inlretro.exe -s scripts\inlretro_inlNESfwupdate.lua
Now that it's easier for the host software to detect the firmware
application version can work towards having the scripts automatically
update the device firmware for you.. But still need to implement this..
this is the verion getting flashed on all v2.0N NESmaker kits
v2.3.0 worked for basic functions, but was never shipped
Majority of effort revolved around testing mapper30 boards with the
smaller v2.0N INLretro with the NES connector alone for NESmaker kits.
added linear feedback shift register for test stream data generated
locally on the device. I'm not 100% sure if this is any faster than
pushing the actual data via USB though.. :/ It's plenty fast on the
stm32 nearly instantaneous for 32KByte. But the AVR takes a couple
sec..
Created "stuff" dictionary for things like that were I just want to add
small things and don't want to bother with a whole new dictionary.
Added file verification to the host with files.lua
Have some nes flash algos return post-written data so calling function
can decide if want to retry, fail, etc.
Changed host dictionary calls to assert instead of error because it
really shouldn't continue. I didn't see an error when sending opcode to
wrong dict and caused head banging..
fwupdate permits bytes to be skipped, or force the update. Found that
the fwupdater got assigned different addresses of ram depending on what
all other ram gets allocated to the main application
Some clean up of inlretro.lua
TODO:
host learn and keep track of the connected device.
Needed for ciccom right now, or knowing whether ciccom connection
is even present..
In the end maybe ciccom is better placed in firmware, but for small
transfers of only a few bytes it kinda makes sense to keep on the host.
Pinport gets quite messy with these made up pin names when really all I
want to do is toggle a specific pin on the NES connector. So maybe some
double mappings would actually be okay, need to rethink that..
create different flash modes that either keep going, retry, or error
depending on the goal of the flash operation. Fanout the return value
from flash algos to all of them.
have fwupdate assigned a specific area of ram so the ram pointer doesn't
change between builds. Okay to ignore for now.
Realized can have STM32F070C6 devices execute bootloader by erasing all
the flash or perhaps even just the first word of flash according at
AN2606. This wouldn't work for RB devices though. This could be done
through the bootloader dict
with large number of updates to the linker script (nokeep.ld)
The first 2KByte is dedicated for vector table, usb driver, usb desc
tables, hardfault, dummy handler, and firmware update routines. There
is currently ~700Bytes of free space in that first 2KB. Should be
plenty of space for firmware update routines and other advanced future
features.
The 070RB has 2KByte pages, and 070C6 has 1KB pages, which is the
smallest erase granularity size. So we can't really have anything
smaller than 2KByte on the RB. This leaves 30Kbyte for the
main/application code on the C6 which should be more than enough.
That 30KByte starts with the reset handler fixed to 0x0800_0800 because
we don't want to have to update the vector table.
After the reset handler is the usbFunctionWrite, then Setup routines
which the usb driver calls for incoming/outgoing data. These need to be
in first 64KByte of flash as a 16bit pointer is kept in usb_buff RAM.
Write was put first as it's less likely to change, with Setup following
which is quite large due to all the inlining that's happening inside it
thanks to the compiler.
Perhaps these function locations could be kept at a fixed location. Or
we could make a 'vector table' of our own just before the reset handler.
This may speed things up a bit, but for now it works. Also like the
ability to change these pointers which may be useful for the next update
as the firmware update code will effectively need it's own Setup/Write
functions. So the current pointers can just be updated to call them
instead, and restore originals/new ones through reset.?
This leaves 96KByte of unused flash on the 070RB, don't have any plans
for this yet. Perhaps future updates for all the connectors and
features will require it.
Also added definition for fast ram functions to .data section. Got that
working but not sure when it may be needed..
Need to physically separate them now. Then can focus on erasing &
flashing ourselves.
Added some speed checks to bnrom.lua script that I was testing usb code
with. Was able to verify read/write speeds were no affected by changes
in this commit. Did some testing against older firmware v2.2 though
there does seem to have been a slight slow down on write speeds.
Although, perhaps that's because of the nrom flash verifications that
are also included in this build (but not commited)..?
Deleted shared_usb.h because it was a copy of shared_dict_usb.h
This build_stm .hex does include some NROM flash updates to allow
checking if the last byte programmed successfully because was having
weird problems with that. But not ready to commit all those changes and
they're highly unrelated to this commit.
Now that usb code doesn't use any .data nor .bss need to fully separate
the USB firmware code from the application. Main way to do this will be
to have usb code be effectively entirely interrupt driven.
Thinking the best way to initialize usb will to have the application
code jump to the USB ISR and maybe use some messaging system with the 2
unused usb_buff indexes (4Bytes).
The USB code will include the vector table, so it will point to the
reset handler, but that will point to the application code's reset
handler, just need to make sure that's at a fixed location.
The USB code is just over 1KByte last I checked, so dedicating 2KByte
should be good. Erase granularity is 1 page (1KB on C6, 2KB on RB). So
that will work well. Write protection granularity is 4KByte, but really
we shouldn't need to use write protection as there will always be the
built in bootloader to save a bricked device.
definitions. Still work left on the expansion port though..
Created *_CONN definitions so code doesn't get included for connectors
that aren't present.
Added a NES CPU write that doesn't toggle M2 but not sure if this will
really be needed for MMC2 or not..
Added windows driver package, just have to run InstallDriver.exe to
get drivers installed on windows 10 (and others I believe)
Created dictionaries for all remaining cart connectors.
Nothing useful there yet, just wanted to get the files created
and dictionaries working.
Added bunch of notes to shared_dictionaries to explain how to go
about creating new dictionaries and some opcode details.
Have STM8 cic communications working "CICCOM" to change between H/V
mirroring on new discrete boards. Currently these operations are handled
entirely from the host scripts and opcode/operands are mostly hard coded.
Need to move these to more generic functions in the ciccom dictionary
which will also speed things up moving to the firmware which will speed
things up.
Some changes to mapper 30 script to eat the ines header, and test CHR-RAM
banking.
Some updates to snes flashing operations, still a work in progress to
fully support prior SNES board designs.
Various changes to STM8 SWIM code to make more versatile allowing SWIM
pin to be located effectively on any STM32 GPIO pin. Still haven't
touched an AVR implementation, but made place holders so it can compile
for AVR at least. These SWIM changes aren't heavily tested, mostly just
made sure could flash SOIC-8 STM8 CIC via CIC CLK.
Beginings of JTAG code to configure CPLDs. Currently only tested state
change and scan out reading MachXO-256, 4032/64v, & XC9572/36XL CPLDs
Tested and working on inlretro6 v1.0p, stm adapter, & avr kazzos.
Older devices with flipflops will apply 5v signals to JTAG pins but time
is mostly minimized by keeping signals defaulted low unless actively
changing states or scanning data.
Still need to verify scan in working, probably move TDI/TDO long strings
to buffers instead of 32byte PBJE data array. Also need smarter PBJE
host code to keep track of current state and come up with PBJE register
values without hard coding them..
But things are working fairly well so far with SWIM & JTAG
implementations. Had some issues where I thought jtag pin toggling was
getting optimized out, but I must have simply had the logic analyzer
speed set too low and was missing pin changes that can be as quick as
40nsec with space optimized code. Current inl6 code is ~4400Bytes,
without optimization it's nearly 50% larger at ~6550Bytes..!
Optimizations seem fine in testing and with logic analyzer running at
50Mhz which is good because the GPIO registers are set as volatile so
they better not be getting optimized away!
AVR not yet working, performing low level SWIM operations will require
decent amount of core specific code due to differences in pin driver
styles, timers, cycles per instruction, etc. The fact that SWIM pin
changes based on the board ADDR0, DATA0, EXP0, etc multiplies this low
level code... Thinking about executing SWIM low level drivers from SRAM.
Initialization could include loading these routines to SRAM.
For now just focusing on supporting SWIM on STM cores for SNES boards.
works on both inl6 and original kazzo just fine. Dumping v3 prototype has
a few byte corruptions on inl6, but is fine on original kazzo. The same
bytes often fail, but not consistently. Tinkered with adding delay, but
that didn't help. Also have issue with adapter not dumping properly.
Prob bug with HIGH ADDR on that board need to sort out still. Going to
focus on erasing and dumping next then come back to some of these issues.
Need to get stm32 up and working, currently the usbFunctionWrite causes
device descriptor request to fail on stm32 devices. So need to do some
debugging there which I was expecting..
tested and verified on purple, green, and yellow/orange avr kazzos and
stm32 inlretro6 proto, and stm32 adapter with yellow kazzo board
AVR takes ~17.5sec to dump 256KB -> 1:10 for 1MByte = 14.6KBps
STM takes ~8.5sec to dump 1MByte = 120KBps
STM32 usb driver is far from optimal as it's setup to be minimal with only
8byte endpoint0 to make an effort to align avr and stm. Larger endpoints
and bulk transfers should greatly speed up stm usb transfers
refactored firmware buffer.c and implemented most of the required opcodes
added check that should cover if device isn't ready for a IN/OUT
transfer. Does this by usbFunctionSetup returning zero which causes the
device to ignore the host. Don't think I've got the stm32 usb driver
setup properly to handle this not sure I fully understand Vusb driver
either. Anyway, hopefully it works well enough for now and keep this in
mind if issues crop up in future.
Still haven't implemented usbFunctionWrite, not sure stm usb driver is
setup properly yet either..
build sizes:
avr yellow/orange: avr-size build_avr/avr_kazzo.elf
text data bss dec hex filename
5602 6 674 6282 188a build_avr/avr_kazzo.elf
previous builds of avr code size was ~6.4KB when flashing and dumping was working.
AVR bootloader is 1.7KB taking up majority of 2KB boot sector.
So AVR has 16KB - 2KB boot = 14KB available, using ~44% of non-boot sector
available flash Have 4 buffers defined, and 512B of raw buffer defined so using
~65% SRAM Making pretty good use of the chip just for basic framework.
Not a ton of room for board/mapper specific routines, so will have to keep this
in mind. Creating more generic routines to save flash will come with a speed
hit, but perhaps we shouldn't worry too much about that as devices below
really boost speed without even trying. There is some sizable amount of
SRAM available could perhaps load temporary routines into SRAM and execute
Also have ability to decrease buffer sizes/allocation. Perhaps routines
could actually be store *IN* the raw buffers.. ;)
stm adapter: arm-none-eabi-size -t build_stm/inlretro_stm.elf
text data bss dec hex filename
7324 0 680 8004 1f44 build_stm/inlretro_stm.elf
Currently targetting STM32F070C6 which has 32KB flash, 6KB SRAM
Could upgrade to STM32F070CB in same LQFP-48 package w/ 128KB/16KB
Don't think that'll be of much value though especially with limitation
on connectors for adapter.
So currently don't have user bootloader, only built in ones.
8KB of 32KB avaiable flash = 25% utilization
680B of 6KB available sram = 11% utilization
32KB device doubles amount of available flash compared to AVR, although
stm32 code isn't quite a condensed compared to AVR.
stm inlretro6: arm-none-eabi-size -t build_stm/inlretro_stm.elf
text data bss dec hex filename
6932 0 680 7612 1dbc build_stm/inlretro_stm.elf
Mostly limited to STM32F070RB as choosing device requiring XTAL, and
desire large number of i/o. This device provides 128KB flash, 16KB SRAM
Currently using 7.6KB/128KB flash = 6% utilization
Currently using 680B/16KB SRAM = 4.1% utilization
LOTS of room for growth in this device!! Part of why I choose it over
crystalless 072 version, as it came with more flash for less cost.
Also hardly making use of 1KB of USB dedicated SRAM:
32B buffer table entries
16B endpoint0 IN/OUT
48B of 1024B available = 4.6% utilization
Have separate lua modules now in scripts/app folder
Dictionary calls are now their own lua module
firmware now capable of calling multiple different dictionaries
have firmware & lua io and nes dictionaries, able to detect
NES and famicom carts. Created expansion port abstraction so most kazzo
versions behave identically.
Created separate make file for stm adapter and inl6
added PURPLE_KAZZO and GREEN_KAZZO defines back in. They work well enough
for sensing NES vs famicom carts so far. GREEN_KAZZO requires
PURPLE_KAZZO to also be defined. GREEN_KAZZO is also only compatible with
AVR_CORE due to software_AHL/AXL functions specifically written for AVR.
I think things will work if a STM_ADAPTER is placed on a PURPLE_KAZZO and
both those defines are made as only real difference is software tying of
AXL and X_OE. But haven't tested this aside from ensuring it compiles.
Have correction to pinport_al.h that will commit immediately after this.
Effectively deleted old dictionary call function/files.
Created lua_usb_vend_xfr function so lua can directly send and receive
vendor setup transfers.
Dictionary calls are more like function calls now, and all args aren't
required so the LED can be turned on for example in lua like so:
dict_pinport("LED_ON")
general format is:
dict_name( opcode, operand, misc, datastring )
Also added ability to store opcode's return length in shared dict library
files with RL=number in the comments following the opcode.
Negative numbers designate OUT transfers, positive for IN.
Default value can be determined by each dictionary's calling function.
Decided pinport is 1 for SUCCESS/ERROR CODE.
Also have default return data means with second byte giving length of
return data in bytes that follows.
dictionary call function reports any errors reported by the device and
returns any return data from the device excluding error code / data len
Now time to start implementing some of these dictionaries on the device.
packet arrives. Had issue with return data on STM32 not being properly
aligned when the rv array was only 8bit. So defining it as a 16bit array
and then pointing a 8bit pointer to it seems to be an easy fix for now.
Ready to start working on pinport dictionary. Need to get lua code
working on a lower level handling the dictionary calls. Need it do do
things like fill out the wLength and everything for me so one doesn't have
to remember every detail about an opcode/dictionary before calling it.
Realizing code was heavily segmented based on how big/many operands there
were and how big the return data was. This is hard to maintain, need lua
to resolve this issue, and make everything easier to script. Thinking
opcode/dictionary calls need to be more like a function call. Passing in
necessary args only, and returning data instead of succeed/fail.
underlying hardware/mcu. Created avr_gpio.h to define AVR pin registers
in a struct fashion similar to what's common with ARM code. Doing that
makes things much easier to abstract in pin macro 'functions'.
Added define to Makefiles that flags pinport_al.h which board is targetted
for build.
Tested and able to turn on/off and pull-up LED on all 3 builds.
Two different Makefiles, specify which with -f file flag:
make -f Make_avr clean program
make -f Make_stm clean program
made release dir to put released .hex firmware files
Need to make separate avr build folder
Need to make one master Makefile that calls one of the other makefiles as
instructed.
Currently device is recognized by PC but does nothing else other than
being recognized by app during connection process:
arm-none-eabi-size -t build_stm/inlretro_stm.elf
text data bss dec hex filename
1332 0 20 1352 548 build_stm/inlretro_stm.elf
1332 0 20 1352 548 (TOTALS)
avr-size avr_kazzo.elf
text data bss dec hex filename
1496 2 43 1541 605 avr_kazzo.elf
enumeration with host, no vendor/class requests handled.
move avr builds into avr_release dir
move original source files into source/old for future reference.
avr-size avr_kazzo.elf
text data bss dec hex filename
1496 2 43 1541 605 avr_kazzo.elf
usb_Func_write updates buffer status if bytes remaining is zero.
Not the best solution as a buffer could be over/under run, define
MAKECHECKS to have buffer mark itself if full.
This method is faster and we always have transfer sizes match buffer sizes
anyway.
Had to add check to get cur_buff status and wait to send payload until
it's empty. Still need to add timeout check as it'll spin forever if
there is a problem and it's never empty...
device should be able to handle buffer sizes smaller than usb transfer
but this probably isn't true if the first two bytes are stuffed into setup
packet. Currently relies on end of (upto) 8 byte transfer to fill buffer.
MAKECHECKS would verify we don't overflow buffer.. Still kind of a half
thought out idea unfortunately.
Not sure how I thought flash operations were previously working as there
were many bugs I had to correct to support flash operations properly.
Operations module appears to be working so far, still need to pass
functions to operation module.
Flash operations verify PRG-ROM 32KB writes working with file comparison.
Currently dependent on extra buffer status reads to delay next buffer.
I think the write operation is taking longer than the usb load operation.
Potentially due to slow code of operation module, but also possible I
had only been testing with slow eeepc linux machine previously. Perhaps
combination of both.
Still need to correct issue so added buff status delays aren't needed.
buffer manager should be able to key off of status==USB_FULL but that
doesn't seem to work. When trying I don't always get the same number of
buffers to get flashed so appear to have a race condition or something
not properly intialized..?
Need sort out sending of USB STALL if buffer isn't ready to be loaded yet.
This commit is mainly for documentation/reference purposes as things are
kind of working, but buggy/unstable.
AVR Memory Usage
----------------
Device: atmega164a
Program: 6486 bytes (39.6% Full)
(.text + .data + .bootloader)
Data: 679 bytes (66.3% Full)
(.data + .bss + .noinit)
Things appear to be working with some early testing. Assumption that oper_info elements
are aligned in SRAM linearly appears to hold true. Researching this I found it probably
was true, but can't be certain esp if gets changed in the future to not be purely 8byte
sized elements.
Still need to provide means to decode function numbers info function pointers.
Prepended DICT_ to dictionary names to prevent using those defines for something else accidentally
"NES/SNES" etc could be used in a lot of places, don't want to use wrong enum/define in wrong place.
created enums.h to list out all enums/defines for cartridge and memory elements in one location.
separate file.c/h file for getting data in/out of a files, and opening/closing them.
adding test roms to roms folder so they can be used for various testing.
buffer opcode updates to transfer payloads
including stuffing two bytes of write transfers in setup packet.
Calling specific buffers with miscdata or opcode.
new dump and flash modules for firmware.
new buffer function update_buffers called during main to monitor and
manage buffer objects when not being loaded/unloaded from USB.
Trying to prevent transfer from exceeding buffer size.
Also verifying buffer's status is properly set to enforce upholding of the status.
Giving usbFunctionWrite a means to communicate it's error/success back to host with USB 'dictionary'.
Had a good lesson on what static means... :/
everything working now as previously designed
speed testing on windows10 PC yeilded ~21KBps when transferring 128-512KB
payloads and 128Byte transfer size. Going to bump to 256 and see how that
does after 128KB speed tests on linux machine.
created host test.c/.h file for general testing of new features.
that way I can start working on erase/write.h files and just use test.c as
scratch code space for tinkering and still call with -t flag on command
line.
modified dictionary calls to include pointers to data and lengths.
moved all buffer operations out of usb.c with new bridge function between
the two files. Lots of pointing going on and lessons learned..
Thankfully everything seems to be working if you actually call the
functions as I designed them.. Gotta love trouble shooting bugs that
don't exist.. Helped updating allocate output to get returned as error
back to the host.
Moved typedef structs to firmware type.h file as seemed to cause
compilation issues being contained in the files .h file when other .c
files needed those types.
Fixed casting warnings with usbMsgPtr ended up looking at usbdrv.c figured
out how close I got, just shouldn't have been putting the * in there..
complete. should be able to allocate buffers from host, but haven't got
to testing it yet. Compiling on firmware though..
Currently have 256 bytes of raw_buffer, and 8 buffer objects/structs
each with ~16 bytes per object. So could trim things down, but still have
decent amount of SRAM left. Could have another 256 byte buffer at this
rate.. but might not leave enough SRAM for temporary routines.
Possible that raw buffer space could be dynamically allocated
as either buffer space or temporary routine space...
AVR Memory Usage
----------------
Device: atmega164a
Program: 4094 bytes (25.0% Full)
(.text + .data + .bootloader)
Data: 573 bytes (56.0% Full)
(.data + .bss + .noinit)
logic 1 if relying on it. Seems to work fine on NES discrete and
INLXO-ROM boards where planning to utilize it. SNES can't pull up due to
pulldown and original famicom cart can't either perhaps because of
EXP6 EXP FF output being too much of a load..?
cleaned up firmware pinport.h used PURPLE/GREEN KAZZO for all #ifdef's
AVR Memory Usage
----------------
Device: atmega164a
Program: 3404 bytes (20.8% Full)
(.text + .data + .bootloader)
Data: 53 bytes (5.2% Full)
(.data + .bss + .noinit)
final kazzo design hex commited as comparable build to the past two commits of green/purple.
So this .hex commit is apples to apples with recently commited purple/green .hex commits.
Able to read PRG-ROM flash chip's manf and device ID from commandline.
New dictionaries io and nes along with firmware files to support.
now have io_reset, nes_init, and snes_init io.c functions
nes.c functions including discrete_exp0_prgrom_wr and emulate_nes_cpu_rd.
New dictionary.c/.h for host to make dictionary calls easier including
setting proper return data lengths based on opcode.
adding nop command to pinport.h
AVR Memory Usage
----------------
Device: atmega164a
Program: 2960 bytes (18.1% Full)
(.text + .data + .bootloader)
Data: 53 bytes (5.2% Full)
(.data + .bss + .noinit)
new file: host/source/erase.c
new file: host/source/erase.h
creating erase files called from main with e_flag
modified: firmware/source/usb.c
introducing switch statement with range of values of opcode type detection.
modified: host/source/usb_operations.h
turning off debug
modified: shared/shared_errors.h
renumbering some error codes
thinking I'll have codes which can come from avr set to 128+
rearranged some of the .h files, created dictionary file to list all dictionaries of opcodes.
moved error codes to shared file so host can interpret firmware error codes.
created firmware usb.c/h to handle usb operations (didn't move as git seems to think..)
cleaned up fw main function and file.
host usb_operations, created USBrequest struct type to more easily handle all transfer info.
Currently able to send pinport commands and read back return values from retro prog.
Just need to start writting functions to send opcodes and start actually preforming some cartridge operations.
Paul Desktop ASUS-C7H