NES scripts to use new dump/flash methodology that MMC3 started.
Includes BNROM, UNROM, MMC1, and new scripts for FME7 & MMC4 (SOP flash).
Adding more general support to SNES with v2proto_hirom that script is
actually becoming more of a master script supporting both LoROM and
HiROM including flash, dump, and save backups.
SNES Rd/Wr now designate the state of /ROMSEL so have to manually
determine if access should be in /ROMSEL space of the SNES memory map or
not. (ie all SNES cart memories are /ROMSEL space except HiROM SRAM).
I made a commit earlier this week but messed things up by not pulling
from the master and things weren't updated... Here's the notes from
that commit:
Author: Paul XPS <paul@infiniteneslives.com>
Date: Tue Nov 6 22:45:52 2018 -0600
Large commit biggest feature add is NES MMC3 support including Save RAM
(aka WRAM/PRGRAM) support including dumping and writing save files.
The MMC3 script & method of dumping/flashing is the most forward
thinking script/firmware so far. Finally starting to form a clear
vision of how I want to handle flashing/dumping variations with mappers.
Biggest thing is having the host handle the mapper init & banking
control. In the view of the firmware, it's only responsible for knowing
how to flash a bank. And dumping is even more generic with the host
just telling what address range to read. Things will get more complex
with support of mappers with bus conflicts. But ready to start
converting these old hacked methods to be more like the MMC3 means.
Have some early support for dumping gameboy using the snes script as the
pinouts are nearly identical. Along with testing of toggling between 3v
GBA and 5v DMG.
Have some early support for INLretro NES only version which uses a
smaller mcu because it doesn't need to support large 16bit carts. Still
have to get this completed.
Added support for CNROM, but I'm not sure if it's actually working. Going
to restart with NROM and start updating the currently supported
mappers to be more like MMC3.
But this also includes some new updates from the second half of the week:
Started updating existing NES scripts to use new MMC3 methodology. Got
NROM, CNROM, BNROM, & Color Dreams working. On the host side only
needed to add nes.c functions for specific flash algos. Able to delete
significant amounts of mapper specific code from flash.c
Got some basic SNES script support with new methodology for Catskull
elect 5v PLCC SNES LoROM board. And INL SNES HI/LO-ROM 3v board as
well. These don't yet use buffer writes, just single byte writes.
Also having issues with Mirroring test/sensing again. Driving me crazy,
but don't really care about it at the moment and not sure what's wrong..
So just committing that broken for now. Looking to remove this
functionality from the firmware side as the host should be controlling
most of this.
Looking to add SNES RAM & buffered writes. Also need to test some of
the HIROM code as I just added it in there while I did the LOROM stuff..
Added windows driver package, just have to run InstallDriver.exe to
get drivers installed on windows 10 (and others I believe)
Created dictionaries for all remaining cart connectors.
Nothing useful there yet, just wanted to get the files created
and dictionaries working.
Added bunch of notes to shared_dictionaries to explain how to go
about creating new dictionaries and some opcode details.
Have STM8 cic communications working "CICCOM" to change between H/V
mirroring on new discrete boards. Currently these operations are handled
entirely from the host scripts and opcode/operands are mostly hard coded.
Need to move these to more generic functions in the ciccom dictionary
which will also speed things up moving to the firmware which will speed
things up.
Some changes to mapper 30 script to eat the ines header, and test CHR-RAM
banking.
Some updates to snes flashing operations, still a work in progress to
fully support prior SNES board designs.
Mostly adding support for mappers as I needed it for my own hardware
builds:
-MMC1
-mapper 30
-easy NSF (still need to update for mapper verilog fix)
-action53 (still need to update for mapper verilog fix)
-dual port board flashing
-colordreams, not sure if I actually got this working
-color ninja, just a special CPLD version of colordreams for ninja boards
Just started working on SNES code. slowly getting things up and working
outside of main inlretro.lua script similar to how NES has been handling
everything with it's own script. Able to flash v3 boards fine. v1 boards
flash without errors, but still having some mapping problems where it
verifies but won't boot. v2 prototype flashes most bytes but not all,
seems v2 boards are much slower to output valid data.. But that may just
be the manufacturer ID codes..?
TODO next:
-bootloader dictionary that jumps to bootloader so don't have to manually
close jumper on the board.
-turn on the watchdog timer for stm32
-create some sort of host timeout so reset button on programmer isn't as
useful
-allow firmware programing algos to be uploaded and executed from SRAM for
faster code that also doesn't require specific firmware builds to support
new mapers.
-Finish JTAG to simplify programing NES & SNES CPLDs
-Sort out swim issue with stm8s001 CICs
-add SWIM support for avr
Dumped/flashed MMX on SNES v3 boards.
created discrete_exp0_mapper_wr to try and write to mapper but not flash
for discrete boards but doesn't seem to work for some reason so commented
out for now.
Most progress was on jtag lua statemachine code. From what I recall I
tested and verified most state change possibilities with logic analyzer.
So they should be fairly good. Possible I didn't test all later ones,
or things are partly unfinished, but my best guess is they're good.
Appears was able to erase MachXO CPLD. Added time delay for run test.
Did some basic testing for gameboy power switching circuit.
Also just got STM8S001 CIC programming working for discrete boards via
A0. Pretty sure I broke EXP0 in the process for SNES boards.. So need
to go back and fix that I think due to new means of changing swim pin.
Various changes to STM8 SWIM code to make more versatile allowing SWIM
pin to be located effectively on any STM32 GPIO pin. Still haven't
touched an AVR implementation, but made place holders so it can compile
for AVR at least. These SWIM changes aren't heavily tested, mostly just
made sure could flash SOIC-8 STM8 CIC via CIC CLK.
Beginings of JTAG code to configure CPLDs. Currently only tested state
change and scan out reading MachXO-256, 4032/64v, & XC9572/36XL CPLDs
Tested and working on inlretro6 v1.0p, stm adapter, & avr kazzos.
Older devices with flipflops will apply 5v signals to JTAG pins but time
is mostly minimized by keeping signals defaulted low unless actively
changing states or scanning data.
Still need to verify scan in working, probably move TDI/TDO long strings
to buffers instead of 32byte PBJE data array. Also need smarter PBJE
host code to keep track of current state and come up with PBJE register
values without hard coding them..
But things are working fairly well so far with SWIM & JTAG
implementations. Had some issues where I thought jtag pin toggling was
getting optimized out, but I must have simply had the logic analyzer
speed set too low and was missing pin changes that can be as quick as
40nsec with space optimized code. Current inl6 code is ~4400Bytes,
without optimization it's nearly 50% larger at ~6550Bytes..!
Optimizations seem fine in testing and with logic analyzer running at
50Mhz which is good because the GPIO registers are set as volatile so
they better not be getting optimized away!
-Updated STM devices to always run @ 48MHz
Doesn't seem to cause any problems with SNES flashing couple thousand SF2
boards have been flashed with this build without issues
-Added note to usb_operations.c as manf/prod ID can't be read if drivers
aren't installed. Caused issues for Todd as he hadn't installed drivers
for new hardware.
-STM swim operations are working pretty well for SNES v2 and v3 boards
Haven't even touched SWIM on AVR core yet...
SWIM is pretty pin independent but only implemented on EXP0 so far
Reads "ROTF" aren't bullet proof but they're pretty good. Biggest
room for improvement aside from adding a legit pullup would be to have
an interrupt trigger the device header bit falling edge instead of the
current polling method which has decent amount of jitter.
Implementing interrupts would also probably allow for more easily
supporing reads longer than a single byte...
Only reads one byte, but good enough.. to get things done.
Code should actually work for low and high speed, but have only tested
high speed on writes so far.
Having issue where reads can fail at times. Esp with long strings of
'0'.. Perhaps operating at high speed would improve matters..
Although I'm also realizing maybe I'm not waiting for the device to reset
and reload HSI trim factory value, need to check that..
The new assembly file/function does everything needed so can start cutting
out inline assembly from swim_out function.
Swim code needs to run at 48Mhz. Realizing this is pretty vital to having
enough time to handle high speed. And timing of artificial pull-up
requires high trimmability..
Able to enter active mode and Write on the fly.
Simple test to toggle LED on STM8 GPIO works!
Still quite far from ideal setup. Some things needed:
-defines for ACK/NAK/NO_RESP in dictionary to report inteligbly to lua
-move test SWIM code into separate lua script
-define STM8-CIC registers for easier calling from lua
-entering active mode is too board dependent, need to use swim_base
-Need to make better use of device timers for entering active mode
-arm assembly is quite a mess, unaware of calling convention when writting
-stopping more than just r0-4, r5+ need to be restored if used
-thinking I'd like a full out assmebly file that gets compiled separately
-nothing is done to support SWIM with AVR
-hacking lack of powerful enough pullup on SWIM pin
not much that can be done to get around this...
don't want to add resistors to programmer for every pin I 'might' use
don't want to add resistors to each board that's made
-seems to work well enough, but reads will prob prove challenging
-currently only running at slow speed with ton of NOPs
AVR not yet working, performing low level SWIM operations will require
decent amount of core specific code due to differences in pin driver
styles, timers, cycles per instruction, etc. The fact that SWIM pin
changes based on the board ADDR0, DATA0, EXP0, etc multiplies this low
level code... Thinking about executing SWIM low level drivers from SRAM.
Initialization could include loading these routines to SRAM.
For now just focusing on supporting SWIM on STM cores for SNES boards.
works on both inl6 and original kazzo just fine. Dumping v3 prototype has
a few byte corruptions on inl6, but is fine on original kazzo. The same
bytes often fail, but not consistently. Tinkered with adding delay, but
that didn't help. Also have issue with adapter not dumping properly.
Prob bug with HIGH ADDR on that board need to sort out still. Going to
focus on erasing and dumping next then come back to some of these issues.
tested and verified on purple, green, and yellow/orange avr kazzos and
stm32 inlretro6 proto, and stm32 adapter with yellow kazzo board
AVR takes ~17.5sec to dump 256KB -> 1:10 for 1MByte = 14.6KBps
STM takes ~8.5sec to dump 1MByte = 120KBps
STM32 usb driver is far from optimal as it's setup to be minimal with only
8byte endpoint0 to make an effort to align avr and stm. Larger endpoints
and bulk transfers should greatly speed up stm usb transfers
refactored firmware buffer.c and implemented most of the required opcodes
added check that should cover if device isn't ready for a IN/OUT
transfer. Does this by usbFunctionSetup returning zero which causes the
device to ignore the host. Don't think I've got the stm32 usb driver
setup properly to handle this not sure I fully understand Vusb driver
either. Anyway, hopefully it works well enough for now and keep this in
mind if issues crop up in future.
Still haven't implemented usbFunctionWrite, not sure stm usb driver is
setup properly yet either..
build sizes:
avr yellow/orange: avr-size build_avr/avr_kazzo.elf
text data bss dec hex filename
5602 6 674 6282 188a build_avr/avr_kazzo.elf
previous builds of avr code size was ~6.4KB when flashing and dumping was working.
AVR bootloader is 1.7KB taking up majority of 2KB boot sector.
So AVR has 16KB - 2KB boot = 14KB available, using ~44% of non-boot sector
available flash Have 4 buffers defined, and 512B of raw buffer defined so using
~65% SRAM Making pretty good use of the chip just for basic framework.
Not a ton of room for board/mapper specific routines, so will have to keep this
in mind. Creating more generic routines to save flash will come with a speed
hit, but perhaps we shouldn't worry too much about that as devices below
really boost speed without even trying. There is some sizable amount of
SRAM available could perhaps load temporary routines into SRAM and execute
Also have ability to decrease buffer sizes/allocation. Perhaps routines
could actually be store *IN* the raw buffers.. ;)
stm adapter: arm-none-eabi-size -t build_stm/inlretro_stm.elf
text data bss dec hex filename
7324 0 680 8004 1f44 build_stm/inlretro_stm.elf
Currently targetting STM32F070C6 which has 32KB flash, 6KB SRAM
Could upgrade to STM32F070CB in same LQFP-48 package w/ 128KB/16KB
Don't think that'll be of much value though especially with limitation
on connectors for adapter.
So currently don't have user bootloader, only built in ones.
8KB of 32KB avaiable flash = 25% utilization
680B of 6KB available sram = 11% utilization
32KB device doubles amount of available flash compared to AVR, although
stm32 code isn't quite a condensed compared to AVR.
stm inlretro6: arm-none-eabi-size -t build_stm/inlretro_stm.elf
text data bss dec hex filename
6932 0 680 7612 1dbc build_stm/inlretro_stm.elf
Mostly limited to STM32F070RB as choosing device requiring XTAL, and
desire large number of i/o. This device provides 128KB flash, 16KB SRAM
Currently using 7.6KB/128KB flash = 6% utilization
Currently using 680B/16KB SRAM = 4.1% utilization
LOTS of room for growth in this device!! Part of why I choose it over
crystalless 072 version, as it came with more flash for less cost.
Also hardly making use of 1KB of USB dedicated SRAM:
32B buffer table entries
16B endpoint0 IN/OUT
48B of 1024B available = 4.6% utilization
Have separate lua modules now in scripts/app folder
Dictionary calls are now their own lua module
firmware now capable of calling multiple different dictionaries
have firmware & lua io and nes dictionaries, able to detect
NES and famicom carts. Created expansion port abstraction so most kazzo
versions behave identically.
Created separate make file for stm adapter and inl6
added PURPLE_KAZZO and GREEN_KAZZO defines back in. They work well enough
for sensing NES vs famicom carts so far. GREEN_KAZZO requires
PURPLE_KAZZO to also be defined. GREEN_KAZZO is also only compatible with
AVR_CORE due to software_AHL/AXL functions specifically written for AVR.
I think things will work if a STM_ADAPTER is placed on a PURPLE_KAZZO and
both those defines are made as only real difference is software tying of
AXL and X_OE. But haven't tested this aside from ensuring it compiles.
Have correction to pinport_al.h that will commit immediately after this.
Effectively deleted old dictionary call function/files.
Created lua_usb_vend_xfr function so lua can directly send and receive
vendor setup transfers.
Dictionary calls are more like function calls now, and all args aren't
required so the LED can be turned on for example in lua like so:
dict_pinport("LED_ON")
general format is:
dict_name( opcode, operand, misc, datastring )
Also added ability to store opcode's return length in shared dict library
files with RL=number in the comments following the opcode.
Negative numbers designate OUT transfers, positive for IN.
Default value can be determined by each dictionary's calling function.
Decided pinport is 1 for SUCCESS/ERROR CODE.
Also have default return data means with second byte giving length of
return data in bytes that follows.
dictionary call function reports any errors reported by the device and
returns any return data from the device excluding error code / data len
Now time to start implementing some of these dictionaries on the device.
packet arrives. Had issue with return data on STM32 not being properly
aligned when the rv array was only 8bit. So defining it as a 16bit array
and then pointing a 8bit pointer to it seems to be an easy fix for now.
Ready to start working on pinport dictionary. Need to get lua code
working on a lower level handling the dictionary calls. Need it do do
things like fill out the wLength and everything for me so one doesn't have
to remember every detail about an opcode/dictionary before calling it.
Realizing code was heavily segmented based on how big/many operands there
were and how big the return data was. This is hard to maintain, need lua
to resolve this issue, and make everything easier to script. Thinking
opcode/dictionary calls need to be more like a function call. Passing in
necessary args only, and returning data instead of succeed/fail.
Had to add check to get cur_buff status and wait to send payload until
it's empty. Still need to add timeout check as it'll spin forever if
there is a problem and it's never empty...
device should be able to handle buffer sizes smaller than usb transfer
but this probably isn't true if the first two bytes are stuffed into setup
packet. Currently relies on end of (upto) 8 byte transfer to fill buffer.
MAKECHECKS would verify we don't overflow buffer.. Still kind of a half
thought out idea unfortunately.
Not sure how I thought flash operations were previously working as there
were many bugs I had to correct to support flash operations properly.
Operations module appears to be working so far, still need to pass
functions to operation module.
Flash operations verify PRG-ROM 32KB writes working with file comparison.
Currently dependent on extra buffer status reads to delay next buffer.
I think the write operation is taking longer than the usb load operation.
Potentially due to slow code of operation module, but also possible I
had only been testing with slow eeepc linux machine previously. Perhaps
combination of both.
Still need to correct issue so added buff status delays aren't needed.
buffer manager should be able to key off of status==USB_FULL but that
doesn't seem to work. When trying I don't always get the same number of
buffers to get flashed so appear to have a race condition or something
not properly intialized..?
Need sort out sending of USB STALL if buffer isn't ready to be loaded yet.
This commit is mainly for documentation/reference purposes as things are
kind of working, but buggy/unstable.
AVR Memory Usage
----------------
Device: atmega164a
Program: 6486 bytes (39.6% Full)
(.text + .data + .bootloader)
Data: 679 bytes (66.3% Full)
(.data + .bss + .noinit)
Things appear to be working with some early testing. Assumption that oper_info elements
are aligned in SRAM linearly appears to hold true. Researching this I found it probably
was true, but can't be certain esp if gets changed in the future to not be purely 8byte
sized elements.
Still need to provide means to decode function numbers info function pointers.
Found bug with setting map_n_part due to >/< instead of >=/<= for setting called_buff...
Was also setting mem_type and part backwards in dump.c
The had issues with usb timing out for more than 1 buffer read back
Problem was due to lack of usbPoll while dumping during double buffering
Adding usbPoll to page read to correct issue
Appears to be issue with dumping first byte of this choplifter cart I'm testing with.
Not so certain it's my bug though.. No matter what I do the first byte reads
back 0x78 and copy I downloaded has 0x00. Setting my first byte to 0x00 also
creates proper CRC32 according to bootgod's database. So need to look into this more
to figure out what's going on.
Detecting mirroring code working and tested
Started working on buffer operations from host
Current code compiles but not yet at point where can start testing
Adding cpu page read to nes.c to have faster dumping operations.
moving enums to shared as gets used quite a bit communicating between device and host.
Prepended DICT_ to dictionary names to prevent using those defines for something else accidentally
"NES/SNES" etc could be used in a lot of places, don't want to use wrong enum/define in wrong place.
created enums.h to list out all enums/defines for cartridge and memory elements in one location.
separate file.c/h file for getting data in/out of a files, and opening/closing them.
adding test roms to roms folder so they can be used for various testing.
now able to accept LIBUSB_LOG_LEVEL from commandline to turn on/off error
messges at runtime. Also setting level > 0 will print messages during
device discovery and connection. Still need to permit kazzo firmware
version to be provided on commandline to support K flag.
fixing dictionary call typo with semicolon and setting buffer length
to always be provided with function call instead of dictionary call
deciding what it should be based on the opcode.
Adding some speed notes and other speed related discussions to buffer
dictionary.
buffer opcode updates to transfer payloads
including stuffing two bytes of write transfers in setup packet.
Calling specific buffers with miscdata or opcode.
new dump and flash modules for firmware.
new buffer function update_buffers called during main to monitor and
manage buffer objects when not being loaded/unloaded from USB.
Trying to prevent transfer from exceeding buffer size.
Also verifying buffer's status is properly set to enforce upholding of the status.
Giving usbFunctionWrite a means to communicate it's error/success back to host with USB 'dictionary'.
moved all buffer operations out of usb.c with new bridge function between
the two files. Lots of pointing going on and lessons learned..
Thankfully everything seems to be working if you actually call the
functions as I designed them.. Gotta love trouble shooting bugs that
don't exist.. Helped updating allocate output to get returned as error
back to the host.
Moved typedef structs to firmware type.h file as seemed to cause
compilation issues being contained in the files .h file when other .c
files needed those types.
Fixed casting warnings with usbMsgPtr ended up looking at usbdrv.c figured
out how close I got, just shouldn't have been putting the * in there..
complete. should be able to allocate buffers from host, but haven't got
to testing it yet. Compiling on firmware though..
Currently have 256 bytes of raw_buffer, and 8 buffer objects/structs
each with ~16 bytes per object. So could trim things down, but still have
decent amount of SRAM left. Could have another 256 byte buffer at this
rate.. but might not leave enough SRAM for temporary routines.
Possible that raw buffer space could be dynamically allocated
as either buffer space or temporary routine space...
AVR Memory Usage
----------------
Device: atmega164a
Program: 4094 bytes (25.0% Full)
(.text + .data + .bootloader)
Data: 573 bytes (56.0% Full)
(.data + .bss + .noinit)
logic 1 if relying on it. Seems to work fine on NES discrete and
INLXO-ROM boards where planning to utilize it. SNES can't pull up due to
pulldown and original famicom cart can't either perhaps because of
EXP6 EXP FF output being too much of a load..?
cleaned up firmware pinport.h used PURPLE/GREEN KAZZO for all #ifdef's
AVR Memory Usage
----------------
Device: atmega164a
Program: 3404 bytes (20.8% Full)
(.text + .data + .bootloader)
Data: 53 bytes (5.2% Full)
(.data + .bss + .noinit)
final kazzo design hex commited as comparable build to the past two commits of green/purple.
So this .hex commit is apples to apples with recently commited purple/green .hex commits.
Able to read PRG-ROM flash chip's manf and device ID from commandline.
New dictionaries io and nes along with firmware files to support.
now have io_reset, nes_init, and snes_init io.c functions
nes.c functions including discrete_exp0_prgrom_wr and emulate_nes_cpu_rd.
New dictionary.c/.h for host to make dictionary calls easier including
setting proper return data lengths based on opcode.
adding nop command to pinport.h
AVR Memory Usage
----------------
Device: atmega164a
Program: 2960 bytes (18.1% Full)
(.text + .data + .bootloader)
Data: 53 bytes (5.2% Full)
(.data + .bss + .noinit)
new file: host/source/erase.c
new file: host/source/erase.h
creating erase files called from main with e_flag
modified: firmware/source/usb.c
introducing switch statement with range of values of opcode type detection.
modified: host/source/usb_operations.h
turning off debug
modified: shared/shared_errors.h
renumbering some error codes
thinking I'll have codes which can come from avr set to 128+
rearranged some of the .h files, created dictionary file to list all dictionaries of opcodes.
moved error codes to shared file so host can interpret firmware error codes.
created firmware usb.c/h to handle usb operations (didn't move as git seems to think..)
cleaned up fw main function and file.
host usb_operations, created USBrequest struct type to more easily handle all transfer info.
Currently able to send pinport commands and read back return values from retro prog.
Just need to start writting functions to send opcodes and start actually preforming some cartridge operations.
modified: pinport.c
modified: pinport.h
-completing implementation of remaining shared_pinport opcodes
modified: ../../shared/shared_pinport.h
-few opcodes deleted because I decided they were stupid when got
around to implementing them. Set user up for errors and not
useful generally.
modified: source/logic.h
-adding IP and OP definitions to use on DDR registers
-ERROR codes denote what function errored out
modified: ../shared/shared_pinport.h
-changing some opcode names for clarity and lowercasing non-recommended ones
modified: source/pinport.c
-fix bug that recreated cur_AHL_addr and cur_AXL_addr
-create opcode with 8bit operand function
modified: firmware/source/pinport.c
modified: firmware/source/pinport.h
-creating shared_pinport.h which is effectively a dictionary for
pinport opcodes
-file gets copied to host and firmware source dirs when compilied.
-hardware macros had to be renamed to include underscore to
differentiate opcode name from hardware macro.
-pinport.c now is a nice clean switch between opcode name and
macro with all literal numbers removed.
-now don't have to manually track/update opcode numbers between
multiple locations.
modified: firmware/source/io.c
modified: firmware/source/main.c
-updates to add underscore pre-fix to hardware macros.
-shared .h files used in both firmware and host app
modified: firmware/Makefile
modified: host/Makefile
-added shared dependency to copy shared files to source prior to
building
modified: host/source/usb_operations.h
modified: firmware/source/main.c
-removing usb commands (now in shared)
modified: host/include/dbg.h
-adding SUCCESS defintion
modified: host/source/inlprog.c
-calling write file if write arg sent
new file: host/source/write_operations.c
new file: host/source/write_operations.h
-creation of write operations files
-opens file and reads header
-some skeleton comments
-lots of work left to do here
new file: roms/nrom_v_test.nes
-adding NROM test rom file
Paul XPS