Added a bunch of code to free created objects. One notable change was
to return the next token from FreeToken() to cut down on boiler plate
code that kept track of a "consumed" token. Now all that just happens
in FreeToken() and I don't have to worry about that anywhere else.
Also added FreeNode() to free all nodes. This wasn't done before at
all. Like FreeToken(), it will return the next node in the list to
avoid having to track that stuff in the calling context.
This node handles both plain paragraphs as well as block quotes. Single
newlines are turned into spaces, double newlines terminate the block.
Care is taken to not have repeated whitespace tokens in the content,
while keeping the number of spaces.
This struct was just a container for two pointers and just made things
painful. This structure has been removed and the next pointer has been
moved to the Token struct.
Also, add a null check to the calling context after NewLexer().
- Parse code blocks that are wrapped sets of three backticks.
- Added a TT_TRIPLEBACKTICK token.
- Added a length field to the Token struct.
- Added an error node.
- Cleaned up some file read code.
- Fixed the header raw text (don't reuse the same buffer for each node).
Zorchenhimer