TBWikiToDo in 'raw' format
This file contains notes about the status and progress of TBWiki
------
{{TableOfContents}}
Note that some test materials are on a separate wiki at:
http://bird.org/cgi-bin/test.cgi/FrontPage
Before working on this wiki any more, re-read this article by a developer
who has written 7 personal wikis:
* https://borretti.me/article/unbundling-tools-for-thought
* the basic idea is that there are better tools for thought than personal wikis
* but that's OK - tbwiki is a hobby project that is fun to work on
= Next major items to work on =
* support captions
* if caption="foo bar" is an attribute of an image, use figure, and figcaption to surround the image.
* see https://www.w3schools.com/TAGS/tryit.asp?filename=tryhtml_figcaption_css
* MacroCategory.py - show a list of pages in a particular category
* implement category tags, like this:
{{{
{{Category(Tasklist,2023)}}
{{Category(New Feature)}}
}}}
* implement category reference, like this:
{{{
{{ShowCategory(Tasklist)}}
or
{{Category(show:Tasklist)}}
}}}
* Blog processor
* support adding a blog entry
* (done) ExportSite processor
* Search macro:
{{{
{{Search(word or phrase)}}
}}}
* do performance enhancements, possibly using the [[https://pypi.org/project/scalene/|Scalene profiler]]
* add a "blog" processor
* for ideas, see: https://jekyllrb.com/docs/posts/
* save blog pages as: blog-YYYY-MM-DD
* don't support comments
* show index of blog posts, current post
* research blog layouts:
* see [[Blog Layouts and Features]] for ideas
* figure out why I keep getting logged out (is there something wrong with
the cookie expiration date?)
* don't add an extra line when saving.
* use splitlines instead of split to avoid adding an extra line on an edit??
* there appears to be no split here
* do some testing to see where the extra \n gets introduced:
* client side or server side?
* in form population (send from server) or in form response data (sent from client)?
* is there a way to detect a new newline, and remove it?
* grep split tbwiki_engine.py | grep [\\]n
* get css from page area of local wiki
* don't retrieve from /var/www/css
* Support wiki on mobile devices
* (done) fix up mobile theme
* change mobile.css and mobile.footer and mobile.header
* some pages still show very small fonts on my phone
* autodetect mobile phone browser, and use mobile theme automatically
* user UserAgent field
* switch to wsgi - I considered flask, see [[FlaskResearch]]
* should switch to wsgi - I have experience with that
* enhance blog system (see [[TBWikiToDo#Blog]])
* add page lock (to support multi-user)
* add help routine to all processors
- add SystemInfo processor to handle help requests
- with help action item, that calls help with a processor name
- processors only have to return a block of tbwiki markup
* make tables robust
* eliminate save_table_data - placing support in appropriate routines
in the io_handler (tb.handler)
* one of write() or some other class method.
* fix bug 017. See [[BugList]]
* fix bug with edit-in-place on [[QuarterlyData]] table
* make automated test suite for testing page rendering
* fix all known bugs, with all known table types
* make a table test matrix
* table flow control
* for transcription program (ProcessorTranscriber.py)
* make a view form and edit form for the transcriber database
* read documentation on view and edit forms
* add processor_name to the transcriber database config
* add support for hybrid tables
* use case is to add a spreadsheet formula to the FIXTHIS pages, to show totals
* ability to have tables with records from different sources
* add io_handler per-record
* need permission checks (readable, writable) per-record
= Things to do (random ideas, un-triaged) =
* don't allow caching a page edit page
* use http meta cache-control directive to prevent caching
* set caching relatively low anyway
* add page locking for edits - to prevent collisions
* lock a page for 15 minutes when an edit is started
* create a lock file in the admin directory
* put user, start time, and page hash in lock file
* start a timer to handle lock timeout
* generate a hash of the page when the edit starts
* if someone else tries to edit, give them a warning that they have lower priority - indicate who has the lock
* when someone submits a page, compare the page hash and lock
* if page hash doesn't match, don't save the data
* remove the lock
* after 15 minutes, remove the lock
* support arbitrary field addressability in json data:
* JsonPageFile:field.list_field[n].foo.bar
* fix bug: edit-in-place wipes out conf entries in inline block
* add 'tbclient list' feature
* modify tbclient to send 'list' request
* modify ProcessorDataHandler.py to respond to list request
* add TODAY() function to spreadsheet
* add websed over snapshotted values
* add websed over diff of snapshotted values
* add chart over snapshotted values
* use plot charts?
* add access-controlled content
* the purpose of this is to add a page with contact information for other
test projects, on the fuego wiki
* see https://moinmo.in/HelpOnAccessControlLists
* at the block level, with ProcessorAccessControl
* at the page level, with accesscontrol directive
* put required permissions at top of block
* prevent reading or writing to block (or page)
* add link to page for file column in attrdb and parsed data files (to go to page)
* This is handy for FIXTHIS entries, and for editing
* [[_bug0013]] - can't edit-in-place the buglist
= Major Features that are not complete enough to use =
== User accounts ==
* Creating and administering user accounts
* need to re-test user accounts, with new features (next_num)
* Users can create accounts, but physical access to the server is required to change admin flags, or to remove an account
* may be able to use a special version of attrdb (raw page edit) for user accounts. It wouldn't be great but would get the job done for now.
* is the user edit form not working??
== Table usability ==
* Specific table usability features:
* other usability feature is deciding which links to use for a table (I never use query, and always have to decide between 'edit-in-place' or 'edit')
= BugList, FIXTHIS, Regressions =
* See the [[BugList]] page for bugs that need to be worked on.
* that bug list is from a database. Here is one that just text on a page:
* [[Manual Bug List]]
* See the [[CodeFixthisList]] page for a list of FIX''''''THIS items in the code
* See the [[PageFixthisList]] page for a list of FIX''''''THIS items in the pages
{{{#.ToDoData
||Date ||bug_count||FIXTHIS_count||missing_FEATURES||total||
||2017-03-08||15||xx||xx||=$.bug_count + $.FIXTHIS_count + $.missing_FEATURES||
||2016-08-12||14||94||2||=$.bug_count + $.FIXTHIS_count + $.missing_FEATURES||
||2016-08-02||13||92||3||=$.bug_count + $.FIXTHIS_count + $.missing_FEATURES||
||2011-04-21 ||17||78||1||96||
||2008-10-06 ||17 ||78 ||1 ||=$.bug_count + $.FIXTHIS_count + $.missing_FEATURES||
||2008-02-04 ||13 ||66 (55 cgi-bin+ 11 plugin)||3||82||
||2007-07-25 ||9 ||62 ||3?||74||
||2007-02-06 ||9 ||41 ||5 ||55||
||2006-11-15 ||8 ||35 ||6 ||49||
||2006-10-06 ||8 ||37 ||7 ||52||
||2006-08-25 ||6 ||35 ||5 ||46||
}}}
== Table of combined issues ==
{{{#!Table
source_spec=#:.ToDoData
cols=Date:bug_count:FIXTHIS_count:missing_FEATURES:total
sortby=Date:rev_alpha
is_spreadsheet=1
field=Date
default_value=%(timestamp)s
}}}
----------
== Regression count ==
Periodically, check [[http://bird.org/test/Regression_Test|the regression test page]] to see if things still work as expected.
Here's a table, manually maintained, with regression testing history.
{{{#!Table:regression_history
||date||version||commit||OK||missing output||bad||bad page list||
||2016-08-12||1.6.2||a45cce77||91||2||2||TestMarkupVariables, TestMarkupVariablesDyamicTest||
||2016-07-18||1.5.3||61eb2f1e||39||31||4||TestMarkupVariablesStatic, TestSandbox, tbwiki.footer, tbwiki.header||
}}}
= Other items to work on, by category =
== Style ==
* enhance themes feature
* see [[DocThemes]]
* for an example of self-editability - see http://kernelnewbies.org/navigation_menu
* calculate theme from user account or other variable
* current use of page name is lame
* allow configurable navigation links
* allow configurable toolbox links
* read page dressing from nested pages??
* allow header and footer pages to be HTML or TBWIKI format
* if TBWIKI, allow for sub-page references that are
loaded, parsed and formatted recursively
* tweak mobile.css
* see w3c pages about automatic layout - http://www.w3schools.com/html/html_layout.asp
* switch to MediaWiki page syntax?? (not yet)
* see http://meta.wikimedia.org/wiki/Help:Wikitext
* need to switch bullets handling
* need to switch table parsing
* have css come from page area??? (would allow editing css using wiki)
== General ==
* convert upload to be a processor
* allows putting upload on any page
* control number of files via config
* can specify link(" action=upload&dest_filename=foo") for missing files - to prompt user if file is missing
* upload result should be shown in page message?
* add page rename
* can do create, move content, delete, but that's awkward
* generate tbwiki stats from log
* cache page content, for static content
* change all prints to output routines (to control routing of output)
* allow administrators to view and edit files in the 'admin' directory
using normal tbwiki access (just like regular pages)
* treat it like a MoinMoin overlay directory
* make a mobile viewer
* add ability to grab content at specific intervals
* kind of a combination of crontab and websed
* allows for charting something automatically over time
* make a crontab processor
* present a form for when to run, and action
* make a syntax for block names that are not included in block output
* we have anonymous blocks, hidden blocks, and named blocks
* named blocks show both the name and the content. It would be nice to hide the name
* possibly: #.BlockName ??
* make tbwiki more relocatable (for use on laptop)
* on bird.org:
* move *.cgi scripts from www/cgi-bin to ~/work/tbwiki/cgi-bin
* put link in www/cgi-bin
* add base_dir to *.cgi scripts
* write script for laptop that does:
* downloads latest tbwiki and tbwiki-data
* adjust base_dir in *.cgi files
* adjust base_url in *.cgi files
* adjust plugin symlinks in tbwiki-data dirs
* make unit test system - see [[Unit Test Notes]]
* NOTE - already have ProcessorRegressionCheck.py
* have a page specify a test
* have markup, and html and compare them
* here's one unit test:
{{{
record=bold 1
markup='''bold'''
html=<b>bold</b>
}}}
* make a ProcessorUnitTest.py processor
- read all "UnitTest.*" pages, convert markup to html, compare to html
- report results
* put this categorized todo list in a table?
* change main() in tbwiki_engine.py
* call main when tbwiki_engine.py is called from command line
* make a tbwiki.py command?
* program should have the ability to create a static page from the command line
* e.g. tbwiki.py FrontPage -o FrontPage.html
* program should have the ability to run unit tests?
== Security ==
* check permissions on table write operations
* allow administrators to view and edit files in the 'admin' directory
using normal tbwiki access (just like regular pages)
* treat it like a MoinMoin overlay directory
* have rights affect the displayed table edit links
== Processors ==
* add help routines for all Processors
* add a processor for "select an item"
* used for choosing a group of words to practice
* used for choosing a story to read in scriptures
* present a list of items from a database, that are unselected
* allow a user to select one
* or, select an item randomly from a list of database records
* allow user to select it or keep choosing a different one
* have 3 categories: unselected, in-progress, finished
* database must "state" field?
* processor allows only legal state transitions
* specify legal state transitions (or is this too meta?)
* support mapping routines (e.g. to convert chapters to pages and back)
* ProcessorScriptureLookup.py - maybe should be a macro?
* take a reference, and replace it with the text for the versus
* this should be very handy for church talks, lessons, etc.
=== Blog ===
* add right sidebar with:
* recent posts
* categories
* search bar
* tags
* use individual pages for each post
* create add-post form
* page schema has title, content, date, tags, id
* navigation to previous and next posts
* response area
* show threaded responses with "reply" button on each one
* response form with "post comment button
* show date, tags, response count
* see example: https://lava-block.com/godot-engine-tutorial-making-a-2d-game-ui-mechanics-sound/
* support customizable background image
* ability to drag and drop images to upload to wiki
=== ProgressChart ===
* add support for goal lines to the ProgressChart processor
* handle chart attributes
* make chart_attrs_class, to generate defaults when attrs are missing
* create flat namespace for chart attributes
* translate chart atttributes into call arguments during chart creation
* un-specialize pychartwrapper.py
* chart another data type using same wrapper??
* generic multi-series data chart
* scripture reading chart?
* weight chart
* running chart??
* support data entry and modification for TimsRunning data
* see ProcessorProgressChart.py
* use Add Form for table??
* support current date for date entry??
=== SlideShow ===
* SlideShow
* make sure all nested parsing works
* add sidebar list of slides (section 1 headings), with links to pages
* support show attributes and per-slide attributes (bgcolor, bgimage, margins, etc.)
=== ImageGallery ==
* ImageGallery
* support links=1 - to show individual images in full screen
* autodetect image size for full-screen (standalone) image gallery
* use Javascript to detect browser window size??
* support num_images - to show only some of the images
* gallery has prev and next links on sides (similar to blog navigation)
* support image borders in ProcessorImageGallery
* support link to fullsize image in ProcessorImageGallery
* support displaying the filenames in ProcessorImageGallery
* add javascript zoom functions
== Tables ==
* add sortby=select_order - to specify that text fields should be sorted
in the order that they appear inm the select possible_values list.
* this helps the user control the sort order for things like word-based
priority: ex: very high, high, medium, low, very low
* add messages to table module
* allow for outputting messages about an individual table
* table.add_table_message(msg)
* this would be handy for table-specific error messages
* should have block-specific messages in general
* allow for configuration of table control links
* have user specify which links to show, and where
* use better filenames
* use unique part of filename for record_id
* allow user to pick filename when creating the record
* support edit-in-place for tbwikidb files
* use get_section and set_section
* modify to read/write definition lines
* eliminate difference between record, record_id
* use tbwikidb or attrdb for user database??
* this puts human readable filler on every page, but that's the wiki way
* this allows creating a multi-file database just by creating the template
* this is fragile - human editors will break the pages parseability
* add table query cheat sheet to query table form page
* make prototype family info database on bird.org
* automatically create a User account file on login???
* make MemberXX records for family members
* determine the schema (what fields)
* create a table view with contact information
* table queries and views FEATURE
* support <table>RecordView
* support read-only attribute on whole table
* Support time-sampled database
* schedule a websed operation that grabs data at crontab-like intervals
* or that creates a new record whenever the page is accessed (and schedule something to access the page automatically at periodic intervals)
* support automatic data snapshotting
* write a 'table creation wizard' processor
* ask for field names and types, and create the schema file and cover file (or template file, if its a tbwikidb)
* document table config concepts:
* base config (from original conf block)
* dynamic config (from request form variables)
* query config (from query conf block)
* saved queries
* specify query list in conf block
* each item in list specifies a query conf block
* make backup file when editing table files
* finish all routines for editing a table
* use {{{Table(arg1,arg2,arg3...) notation for inline tables
* support other args:
* formspec, query line (row filter, column filter, etc.)
* document Table use with data_scan module
* [[DocTables]] has some information (one example, the FIXTHIS table)
* multi-file tables
* display multi-line values correctly
* display '<no-data>' values correctly
* auto-sense type of each field (text, select, textarea) and use
for auto-form generation
* handle field types with enumerated values (lists and radio buttons)
* document table architecture and interfaces - see [[Table Architecture]]
* document table attributes
* document chart attributes
* make ProcessorTablePick.py
* See [[Table Pick Page]]
* present simplified list of "unpicked" items
* allow user to pick one item
* save picked item with attribute "state" changed to "in-progress"
* present detailed list of "in-progress" items
* allow user to mark an item as "done"
* save done item with attribute "state" changed to "done"
* whether user wants to have multiple items in "in-progress" state is their choice
* queries, query_lists
== Forms editing and Templates ==
* use form emitter for user accounts.
== Users and attributes ==
* Support online creation of new users
* make a "create user" form
* create a new user account file from a form
* provide help if user forgets password
* link to e-mail the password??
* password hint??
* create 'user.edit' action, which edits a user account file
* support different roles (admin vs. non-admin)
* allow user to edit their attribute values
* make an "edit user" form
* REFACTOR - use existing form code for user forms
* make user account management it's own processor module
* ProcessorUser.py - we now have sub-actions, so this should work.
* it needs to access table operations, but in a special way
= Long term ideas =
* add 'save expected output and show again' to RegressionTest processor
* add link to backup page on history page
* look at other wikis for feature ideas
* in particular, [[http://ikiwiki.info/|ikwiki]] statically compiles the wiki on every page edit
* documentation
* table editing:
* should support "save and edit again" and "save and add another row" buttons for saving edit forms.
* Table Of Contents
* do auto-table of contents, like wikipedia??
* handle included files and text expanded from processors??!!
* check HTML and CSS on multiple browsers with BrowserCam or browsershots
* see http://browsershots.org/http://bird.org/tbwiki/FrontPage
* make a journal processor (based on progress chart)
* make a blog processor (with threaded comments)
* make a forum processor (with threaded and voted-on comments)
* make a page cache
* add dependencies to a makefile, and rebuild cache when dependencies change
* put config into admin page?
* reduce cgi-bin to just some directory references
= Things Completed =
* 2021-03 Remove '?action=save' from url in edit form
* 2018-10 Add image rotate to ImageGallery app.
* 2017-04 add triggerd - ability to detect triggers that are due and execute them
* 2017-03 add ProcessorTrigger - ability to trigger a snapshot
* 2017-03 add ProcessorSnapshot - page/block/table snapshot function
* 2017-02 add tbclient - ability to get,put and edit items in the wiki
* 2017-01 add page trails
* 2017-01 add relative index to cell address (so you can do $.-1)
* 2016-12 ProcessorImageGallery - finish showing images in full size page
* 2106-11 add ability to customize table operation links (edit_links, sort_links)
* 2016-9 add ability to read a specific block, by number
* 2016-08 convert to per-table messages (for debugging and errors)
* 2016-08 support editing sections with nested sub-sections
* 2016-08 allow default_values with generated values (%%(timestamp), %%(next_int), %%(next_page_num))
* 2016-08 add page delete
* 2016-07 add tbwikidb database type
* 2016-07 fix data_scan module to parse multi-line fields
* 2016-05 make ProcessorImageGallery
* 2016-05 make ProcessorBlog
* 2016-03 make better diff output
* 2016-03 put engine_version in data_class, and add to footer
* 2016-03 add "printable format" option for page
* 2016-03 made a test wiki at: http://bird.org/cgi-bin/test.cgi/FrontPage
* 2014-05 make talk pages optional - see config.has_talk_page global config
* 2010-10 support inline tables - FEATURE
* 2010-?? websed - add websed feature to scrape other web pages
* 2010-?? data-scan - add as a processor first, then make into a database grabber
* 2008-10 table - back up page on table edit
* 2008-09 chart - draw chart of table data
* 2008-08 processor - add a charting package (pychart)
* 2008-02 table - added support for sorting table rows
* 2008-02 table - support col_list and row_filter specifiers
* 2008-02 table - added ability to edit whole table at a time
* 2008-02 macro - create Include macro
* 2008-02 table - separated conf into separate class
* add field_db (parsing, handling)
* add field types, colors, possible values
* add routine to read block from page
* 2008-02 table data - separated IO handler into separate class
* move tbwiki with authentication to bird.org
* aug03 - add user accounts.
* put user account files in an 'admin' directory
* authenticate user from files in 'admin' directory in action user.login
* look up user by name to process login form (in action user.login)
* jul07 - add cookie code for user accounts
* feb07 - use pages for page dressing (see tbwiki.header and tbwiki.footer)
* jan07 - support file upload FEATURE
* 11/15 - add support for inline images
* change backup routine to save current version of page instead
of old version of file
* save edit comment and show in history
* edit sections FEATURE
* parse and format:
* bullets
* headers
* horizontal rules
* paragraph breaks
* preformatting
* write edit form for page
* save edited data to file
* read page from data directory and emit as html
* write handlers for various actions
* create add_record form (using existing edit_form code)
* write mtable.save_form() routine
* re-write edit_form to present whole table with edit links
* support Table internal macro, with args
* modified backup time in file extension to be human-readable
* wrote mtable.modify_record() routine
* wrote mtable.remove_record() routine
* parse and display URLs as links
* use pages for page dressin
* make items in table of contents into links
* make an anchor target for each heading
* use href for anchor target in TableOfContents macro
* make nicer format for table of contents box
* add search form to header
* support multi-page table source_spec
* support e.g. "User*" as the spec
* read all pages starting with User
* finish parse_table_pages() routine
* read each page matching the source_spec
* parse out values using a page template
* start with simple name=value parser
* create page on add record
* write page on edit record
* remove page on remove record
* support history action
* show history from backup times
* allow diff between two versions