Wiki Ide

Inspired from ProgrammingInWiki

A 'Wiki IntegratedDevelopmentEnvironment' or WikiIde would be a collaborative development environment for applications and services, and probably suites of such.

The virtues one would expect of any modern Wiki would be part of the WikiIde (if it is to deserve the 'Wiki' name) - these include:

The ability to edit the Wiki in a browser, over the web.
Collaboration - multiple editors.
History. A WikiIde would go further with this, and allow full source control and 'snapshots' - views collected of relevant parts of a Wiki at a given point in time.
AutomaticLinkGeneration or at least 'easy' link generation (e.g. WikiWords).
Discussion, Commentary, Markup - akin to LiterateProgramming, but with HyperText; distinct discussion pages and manual pages associated with each page may also be provided
UserPages, with user information. This would include information of the "I'm gonna blab about myself for a while now" sort, but could easily include a wide variety of other information: images, contact information, public keys to check signed contributions, certificates signed by various project-leads offering rights to administer, search, or access certain medium-security projects, preferences for syntax highlighting, subscriptions and RSS feeds that continuously send detailed updates back, personal projects, and any additional per-user support (e.g. tracking ThreadMode conversations). In the IDE side, UserPages (or a commonly named page under the user's UserPage namespace) would be ideal places to track instantiated processes with throw-away names
Search, at minimum. In its capacity as an IDE and RefactoringBrowser, support for semantically meaningful queries and 'views' of the Wiki contents would be a far better deal. 'View' support would also integrate well with project versioning.
Support for hierarchical namespaces as part of the 'Wiki' would be favored by the 'IDE' aspect. It could be as simple as any page acting as a namespace via some sort of tagging mechanism. This would be similar to 'categories' as provided in a plain wiki, but the flat namespace provided by regular wikis would not readily support the vast supply of redundant names ultimately required for project configuration management, and would create difficulties when a particular project needs to override the meaning of a particular WikiWord (e.g. due to versioning, or due to a name collision). Disambiguation pages could still be provided, linking to a word under every namespace that provides it in addition to the 'root' namespace. Category pages could still be provided, linking into various projects and instances that are tagged with the category identifier.

Similarly, one would expect certain properties associated with programming languages and IDEs:

Semantically significant links (WikiWords) - referencing values, functions, algorithms, procedures, active services, process descriptions, predicates (which may be associated with fairly large databases constitution propositions that fill or explicitly don't fill the predicate), service-clouds, etc. Depending on the context, a reference might be part of a discussion (as per a traditional wiki) or something to-be-applied in some manner or other (e.g. a function in position to be applied over values at 'runtime' or as part of a 'view', a service to be accessed, etc.)
- Note: This suggests, possibly requires, a language with both syntax and semantics designed for use in the context of a Wiki. Such a language could still work outside of such a context, but would still involve hyperlink-style lookups on commands identified by WikiWord.
- Note: There are some levels of support available between traditional languages and the one described here. For example, one could limit WikiWord use to the 'import' statements (i.e. 'import BubbleSort'). This wouldn't be quite as sweet as just writing 'BubbleSort this_list' somewhere in your code and having the compiler figure out that you want to apply the BubbleSort algorithm to the list you offered.
'Project' specification support. A project isn't just an algorithm hanging off on its own; it's a mashup or agglomeration of different pieces of code applied to some unified purpose, such as provision of a service. Essentially, as an IDE, a WikiIde needs to provide 'project' pages that describe the construction of various applications. These would serve vaguely the same purpose as MakeFile?s - i.e. specifying how components are to be linked together, not active runs of the service. Projects would have many associated components, such as project description pages, project administration (user rights management, for example), project versioning (allowing snapshots of the linked WikiWords or components at any given time), project statistics (ideally constructed automatically), project 'test' scenarios, project documentation (e.g. requirements, user-stories, 'cards'), etc.
- It might be possible to apply constraints and aspecting at the project-level, too - e.g. whether strict type-checks should be applied, or whether HardRealTime constraints must be guaranteed, or whether the service or application described by the project is to run in a fixed amount of memory, or where to send the error logs.
- Support for minor tweaks to a project description - i.e. project configuration management - is nearly essential. Something like inheriting a project and overriding various features would be a reasonable way to go. This would allow 'aspected' configurations of a project (with aspect-oriented programming) or to override where error logs go for a particular run of a mini-project designed for exactly that purpose.
Project test support, especially with feedback. Something similar to an REP loop would be very useful, just as it has proven to be in many other IDEs - allowing immediate tests of fragments of projects. UnitTests for fragments of code would also be a major augmentation to project-specific 'test' scenarios.
Support for running projects and interacting with them, ideally in some non-volatile manner (since volatile activities aren't very 'Wiki'-like). This could include active instances of projects (be they services or modal applications) given WikiWord names. This would suggest preference for a ProgrammingLanguage that readily supports persistent and non-volatile processes with small runtime costs, such that they can easily be pulled out of persistent memory only when required, and farmed out to various servers when experiencing heavy use. This, in turn, suggests a language with an explicit ProcessModel - e.g. ActorModel, CSP, or ProcessCalculi - and possibly with FirstClassProcesses? (which would be useful if you can link them via WikiWord names). Non-volatile instances of services and applications implies you could close the browser, go to the movies, go home, log back into the Wiki, open up the instance by some linked WikiWord (even if it is back up, and continue operation.
Support for debugging applications running in the WikiIde.
Support for packaging up projects and creating 'installers' for them that will allow them to run on various other systems - e.g. WikiIdes owned by other people, on Mac, on Windows, on Linux, on bare metal, etc. This implies support for compiling to a number of different back-ends. The mechanism for running on the Wiki in which it was developed should, perhaps, be an instantiation of this package-support that is made readily available for all projects; packaging up for other systems might require reconfiguring the back-end services. (one can essentially treat an OperatingSystem as a back-end in much the same fashion as bare metal - one essentially describes how to access certain required services, even if it requires creating them on the fly).
Support for automatic generation of throw-away Wiki names. These would be useful the initial WikiWord identifiers for instantiated projects and their associated services or applications. For example, when a user instantiates a project, it could automatically get a WikiWord associated with the service/application that not only provides a handle for using it (e.g. go to that page to access the application, or send messages to that WikiWord to interact with that service), but also would provide a handle for any other process information (start time, resource consumption, held capabilities, priority, etc.) and for managing it (kill it, give it an expiration date, redirecting a more permanent WikiWord to it, etc.). Users don't need to worry about naming things (e.g. they just instantiate a project and its throw-away identifier shows up under UserPage:ProcessInfo until such a time as they choose to explicitly destroy the process or remove all links to it. WikiIde-wide garbage-collection becomes possible based on removing processes that aren't referenced from a 'permanent' source. (Of course, there are other mechanisms for specifying collection properties... such as attaching an expiration-date and/or using a WikiWord to identify that a page should be collected.) Users may even control, to a small degree, who else can see the process... but as a security measure, capability-security is a better choice.
RefactoringBrowser for source, ObjectBrowser for projects and especially active values and services. These are 'views' of the Wiki in a sense, but to support particular utilities - e.g. renaming a particular WikiWord across the entire Wiki.

Properties required as a Collaborative Web & Code Services platform:

Access to the running services by a range of protocols. E.g. grabbing port 80 and 443 for HTTP (and maybe something like 2280 for rescue - see RuntimeUpgradeableCore), but also additional ports to support non-HTTP protocols... e.g. something to trap and support WSDL or SCA or CORBA or fast, trusted binary communications with supported services, each with dedicated ports and each built and loaded from within the WikiIde. Keep in mind that the WikiIde services are named by WikiWord pages - i.e. the 'URI' references running services/'objects' far more than the port number.
Safety and Security concerns must be fully addressed. A WikiIde could be used for some phenomenal cross-site scripting attacks, as a platform for negative operations (e.g. spamming), and is subject to programmable virii. There are people of questionable motives who can and will attempt to inject malicious code for fun or profit. Frankly, the WikiIde host cannot reasonably allow for fully anonymous access to the WikiIde for purposes of creating or running projects - there are likely to be legal issues that arise should the WikiIde be utilized to some nefarious purpose. On a per-project basis, also, one needs to be careful that 'bad' code not be mixed into the good stuff. See AssumeGoodFaithLimitations.
- It needs to be made very easy to control access to the WikiIde - in particular, to remove those who might be considered 'vandals', and possibly to legally persecute or track those who have been intentionally causing (serious) harm. (Even the threat of exposure or banning would be a large deterrent from inside-job attacks, and would also help deter license violations and other misbehavior.)
- Integrated use of a WebOfTrust: ability for projects to list 'trusted' parties and have these 'trusted' parties sign off on particular versions of linked pages (with such signatures entering a common 'tags' database). At build-time, then, only the most recent 'trusted' versions of pages will be integrated with the project. Note: it should be possible for projects to 'trust' parties who aren't part of the project team. I.e. I could say that my project trusts any page that some other project trusts, or any page some particular person who isn't on my project has signed off on, which allows me to be a lot lazier about trying to figure out which versions of which pages I actually trust (there isn't a lot of value in massively duplicating effort!). This TrustWeb? would vastly reduce instances of 'malign code' entering a project.
Sane license control without any hidden gotcha's - everything needs to be damn obvious to a programmer. The easiest way to handle this is for the host of the WikiIde to demand that software be provided with a particular license (creative commons, gpl, etc.) or as public domain, with purpose of guaranteeing legal interoperability. But sweeping rules tend to cause problems. Perhaps better: most stuff, excepting objects under a ProjectPage?'s namespace, a UserPage's namespace, and a License namespace, must use a common (very open) license - perhaps disclaiming patents and such, but available for commercial use and compatible with everything. Then stuff in a ProjectPage?'s namespace could be licensed on a per-project/per-page basis. And stuff under a page/namespace like 'GnuGeneralPublicLicense' would be more obvious in their licensing (leaving only the question of version 2 vs version 3). Users would know when they need to be careful about license stuff because they'd be using namespaces that would appear like they are peeking into somebody else's code or appear like license names. Thus, it would be less 'hidden'.
- Of course, figuring out how to properly apply, say, the GnuLesserPublicLicense? to a WikiIde would still cause headaches. Does it mean that commercial systems can only link the final 'object/service' and not the code?
- It wouldn't hurt if some reflection could automatically compute whether a project utilizes any conflicting licenses, and to automatically report the license(s) under which a project-as-a-whole is provided. Brings a whole new meaning to TypeChecking, eh?
- Culturally, use of deviant licenses for code should be discouraged on the WikiIde through peer pressure. Individual licenses would tend to create rifts across which projects cannot communicate, and aren't a GoodThing relative to the WikiWay. However, there are also costs of not allowing programmers control over their own creations, so a decision will need to be made by the wiki hosts on the involved requirements.

Wishlist:

Some ability to extend this set of supported protocols from within the Wiki (e.g. adjusting a WikiPage redirect that determines that WebServer in current use) would be an advantage... e.g. one could add (from within the WikiIde) support for advanced ObjectBrowser and debugger support that needs higher bandwidth or more of a publish/subscribe approach (should HTTP be deemed insufficient).
Support for distributed services, parallel operations, leveraging server-farms; distributed WikiIde
Transactions support

In many ways, a WikiIde is nearly a full-blown operating system. This is hardly a unique property to WikiIde, of course (a great many IDEs, such as those for Lisp, Smalltalk, Ruby, Erlang, etc. are essentially operating systems minus the device drivers). In any case, if we have a WikiIde written in WikiIde such that it can be repackaged for bare metal, it wouldn't be a bad idea to actively explore the possibility of running it on bare metal using a variety of WikiWord services for hardware-access and resource management as well as the browsers themselves. Perhaps it could be reversed, and OS ideas could be added to flesh out the WikiIde (see NewOsFeatures, ObjectBrowser).

Aside from being an interesting curiosity, what would a WikiIde give you that couldn't be achieved equally well (or better - editing code in a browser textbox sounds grim) with the combination of conventional Wikis, Web-based forums, email mailing lists, or other communications tools used with conventional IDEs and a source control management system?

In what way to "conventional IDEs and source control management systems" use the schema of hyperWords and the power and diversity that their linking and architecture provide? What is so grim about editing text in a textbox? It is easy, as this wiki demonstrates. Perhaps there are social and personal issues to be overcome which make implementation of a WikiIde more difficult than the technical issues which it presents. It seems to me that this concept is more than a simple curiosity, and that it is part and parcel of a new wave of thinking and implementation which makes the workplace mobile and virtual rather than fixed and physical. -- DonaldNoyes.20080227.1613.mo6

What "power and diversity" does a "schema of hyperWords" and "their linking and architecture" provide?

This is adequately exposed in the proposal starting at the top of the page with "The virtues one would expect of any modern Wiki" and ending with "the entire wiki" (1400 words)

A page named ProgrammingInWiki discusses potential benefits. Editing in a textbox isn't so bad - I'm sure a little AJAX support could even give you syntax highlighting. Instead of anything technical, I'll perhaps offer some snapshot UserStory(ies) and UserExperience?(s). Feel free to comment. Feel free to contribute if you can come up with something - taking ownership here might close discourse too much. Phrase your own in first person, if you wish.

I'd love to be able to program in a web-browser, from anywhere, and get real work done. I hate being tied to the computer that has installed upon it the expensive commercial IDE. I hate even more attempting to pipe a virtual desktop from one machine to another, which invariably results in delays (even worse: unpredictable delays) between action and response.
Explicit management of source code and revision control is a pain. Even after working with some of the best free alternatives (e.g. SVN), I'm left envious of the simplicity offered by modern Wikis. In MediaWiki, I can just click 'History' tab for any page and view past versions thereof. I want that degree of version-control to be part of my IDE, and available over the web. I'm certain this could be done in some of the nicer existing IDEs, but I don't want to have to think about it or do extra work for it to be there.
At the moment, to do serious work, I often need to have a bunch of different communication sources open or available at once: e-mail, bugzilla, the IDE, possibly IRC or telephone. I'm not particularly keen on adding web-based forums or conventional Wikis to the list. These are problems I want vanished:
- One problem is that none of these are truly integrated with one another - I can't search the forums from my e-mail browser. Even where one can provide hyperlinks (e.g. e-mail linking to forums), these must be written explicitly (no AutomaticLinkGeneration) and there are no automatically generated reverse-links, which are rather nice as features go.
- Many other components (aside from content) are not integrated, either; consider management issues:
  - any security policy I attempt to apply to the e-mail policy is not automatically applied to the forums, the version control, etc. With WikiIde, I could demand capability-security to see or modify or interact with certain pages - just one feature to secure it all.
  - any insurance on the backup of the intellectual labor involved in them goes right out the window: phone stuff is lost anyway, but e-mail, IRC, bugzilla, the forum, the source control, the uncommitted source on user machines, etc. all have independent backup-policies from the wiki. Keeping things 'centralized' management-wise would be a huge advantage here, even if the policy is to securely distribute the wiki to dozens of machines with farms of runtime servers.
- Another problem this panoply of sources has a high resource cost - if I'm going to work somewhere (e.g. while on travel, away from my office machine) I need to (a) get the e-mail working, (b) forward my phone number to a cell, (c) ensure I have a machine with the expensive IDE, (d) ensure my machine has the right version-control software installed, (e) ensure I have copies of any critical e-mail informing me of what I should be working on, etc.
I'd be delighted to have the ability to link bugs and requests directly to the appropriate parts of the code. Not only could I peruse the bugs and dive directly into coding, but I could use reverse links: I can take a piece of code, click on its WikiWord, and find every mention of it in the WikiIde's integrated-equivalent to BugZilla, discussion pages, etc. With a proper ObjectBrowser, I could even limit the results to just those pages with a 'ToDo' tag or whatnot.
I'd rather enjoy seeing a group combining the notions of WikiPedia and SourceForge and creating one ginormous WikiIde for anyone and everyone to create OpenSource projects. It's actually things of this scale for which I believe WikiIde would be best - though, as mentioned, good per-project source control regarding which versions of other code are being integrated locally are of high importance. Currently SourceForge is a bunch of almost entirely independent projects; this would change the formula entirely: projects would be integrated by default, and explicit per-project version-control management would be the only thing that allows them to diverge.
I want to be able to use WikiIde as a mashup-maker, a lot like IBM's QedWiki. I.e. I just create a project-spec that integrates algorithms, code, other services, etc. either already available on the WikiIde or through WikiWords capable of integrating external sources. I put these together in one location, press a button (or not), and voila! a new service is available, running on some hidden location on the WikiIde's integrated server-farm (possibly duplicated and distributed worldwide).
Current IDEs are mediocre at best at handling distributed-programming projects - a truth even when leveraging a language designed for distributed programming (like Erlang). One often needs to port code or binaries and fire up services, by hand, on different machines - often a variety of different machines - to perform a test. If you've never done it, take my word for it: it's a royal pain. With some effort, one can start automating the shifting of code and services to different (often virtual) machines as part of a distributed test, but even then running any sort of debugger is a herculean task. I want this to be easy: I want an IDE that can integrate with whole farms of machines, if needed, and run dozens or hundreds of test instances. This would hardly be unique to WikiIDE... but for a WikiIDE to really work on a large scale, it would be a de-facto required feature. To make it beautiful, it'd just need to be implicit.
A place like SourceForge doing the WikiIde thing won't be able to freely give away arbitrary computation resources (space, cpu) - i.e. some limited amount per user and per project they could handle, but with massive farms of servers, people will want to run some computation-intensive concurrent programs... like full-blown WorldOfFreecraft games (for which nothing is charged to regular players, though renting GuildHalls and advertisement space and such). WikiIde could become a tool for me to sell further computations, since it so readily integrates such things as user-pages, security, project administration, capabilities and such... one could use it to represent non-repudiable capabilities and service-contracts that the server-farm can respect. Alternatively, someplace like SourceForge could provide the WikiIde themselves but allow independent server-farms to compete for these contracts - and they would; if the bar was set low enough and security set high enough, companies would even start siphoning off extra computer cycles on their work machines. Everyone wins. Computation and communication - even extreme computation and communication - would suddenly become available (cheaply, and very readily) to anyone with a web-browser, a decent education, and a wallet that could pay for any modern computer. As the WikiIde would be pretty much on the big servers instead of on the machine you're typing into, you could even get this power from a cell-phone. The ability to perform extremely intense computations in short bursts without paying for a server-farm would be a major boon to a wide variety of industries.
The ability to run final services in what is essentially a permanent IDE has its own considerable advantages. The services can be updated automatically as the connecting WikiWords and services are updated. The WikiIde can track and recompile and re-optimize pieces that have dependencies that change. I can debug my services at any time I wish - e.g. open windows to hook into the state and messages in order to track things.

This rationale makes sense. The justifications [at the top] didn't convince me, - (Technically, the stuff at the top was a list of properties, not a justification for desiring said list of properties. But DonaldNoyes has before expressed himself of the opinion that if hyperWord technology is built, justifications will later be found. He's probably correct.) - but this comes closer. However, I spend much of my day quite happily working in the EclipseIde, I grit my teeth when I have to edit in a browser textbox, and I've written a fair whack of browser-side JavaScript, so I have a very hard time imagining that a browser textbox can provide anywhere near the same speed and functionality, even with (for example) a TiddlyWiki-like festoonment of JavaScript-based enhancement. I suspect the idea will be much more viable (as far as I can see, at least) when browser capability is considerably more advanced than it is now. I'm still skeptical, but I'm starting to like the idea, and would be very interested in exploring the minimal infrastructure necessary to support the essential attributes of a WikiIde. Toward that end, I'd be happy to collaborate on creating a SpikeSolution, and can donate some server resources to trying it. If adequately designed, I believe it should be possible, over time, to evolve the spike into the system described above entirely via the Wiki interface.

ThinkingOutLoud here... I would strive to make the initial kernel, if it can be called that, as minimal as possible, with the goal of moving into a purely Wiki-based editing environment as quickly as possible. This brings to mind the ForthLanguage; it can be ported to a new platform and bootstrapped by implementing only a tiny handful of low-level mechanisms. I would hope a WikiIde would be the same. -- DaveVoorhis

On AJAX and Grim Textbox Interfaces

It wouldn't be impossible to enhance the browsers, too, just to make things easier... i.e. have a Firefox extension for the highlighting or 'improved' view of the Wiki, even if one can get by with the default view. And, frankly, AJAX is something of a weak solution in the long run; if we have get this huge WikiIde running on some awesome language of WikiWords and such, I can't imagine it would be impossible to create a client-side 'scripting' language that simply hooks back into the WikiIde, and eventually use it to replace the "text/java-script" stuff. (Providing browser plugins is relatively easy... the WikiIde could even host them so you can download them wherever you go.)

I've been studying up on AJAX and CSS, thinking about syntax highlighting and mechanisms for implementing a 'sweet' update interface. I'd like to see as follows on the EditPage:

No Textbox. There are wikis where I am able to double-click at some location on a page and just start typing, and WikiIde should do the same. AJAX takes care of delivering the edits. After I start editing, other people should be able to see the edits in a reasonable period, possibly see that the area is being edited, and the versioning software should be able to very readily integrate several different users working on different sections of the same page. Versions are provided, and the versioning software must be designed for efficient versioning of micro-edits. Undo-capability should also be providable via the browser, but is undo-forward (meaning that the 'undo' is actually reflected in the history as well). Collisions, which should be reduced in probability due to the grain of the versioning and the software's ability to merge most edits, will just be handled by further editing. The history-view can collapse micro-edits temporally into large macro-edits.
- It seems that even FlexWiki's double-click-to-edit isn't what I'm looking for, which is the ability to edit inline without opening a textbox.
  - FlexWiki uses "<body onclick="javascript:BodyClick?()" ondblclick="javascript:BodyDblClick?()">" to capture the clicks that forwards one to the proper WikiEdit cgi with the given WikiPage. Ultimately, this allows for a very efficient click-anywhere-to-edit, but has its costs - e.g. it doesn't track -where- you clicked, so your cursor isn't at the correct position immediately after clicking.
  - For inline-editing, one needs to use AJAX to essentially capture and send the updates... and also to report the locations on a page that other people are editing inline, and also to do a certain degree of collision management. However, these features require (fundamentally) the ability to identify the location -where- the user is attempting to edit.
  - I am not sure how much information can be captured regarding mouse location or highlighted area. At the very least, the div-identifier can be captured for editing purposes. I'll continue studying AJAX programming patterns until I come up with a practical approach to this.
    - It seems that this activity is nigh-trivial in a Gecko browser. One simply uses: domObject.selectionStart and domObject.selectionEnd (e.g. as part of a doubleclick). For a simple double-click without selection, selectionStart and selectionEnd are the same. Microsoft's IE is more difficult. See http://parentnode.org/javascript/working-with-the-cursor-position/
- The most fundamental rule for inline editing seems to be that absolutely no characters or codepoints be added to or removed from the view of the source. Every character that is part of the source MUST be visible to the user. There can't be any equivalent to the 'apostraphe' being used for italics or boldface, and there must not be any equivalent to WikiPedia's [[link format]] or {{template formats}}. The display may add characters, but these characters must not appear (to the user OR to the inline-editing tools) as part of the source. For anotations, one can use floating hover-text, popups, underlining, bold-facing, font-choice, font coloring, semi-transparent text that can't be edited, et cetera.
- I believe that WikiIde may be better at fulfilling these rules, as the source IS the real content. (In a regular Wiki, the content is the formatted text, not the source, so displaying the source to everyone would be a pain.)
Be able to highlight a chunk of code and select an option to 'reserve' a section of a page (e.g. a block of lines) for editing. Other users can see that this section is being edited by you via shading or bracketing (and some AJAX). This reservation should be time-limited - if unedited for a few minutes, the current version would be saved (albeit not checkpointed) and the reservation dropped.
- Chat-capability should automatically be added to the reserved part of the page, such that other people can right-click on that section of the page, raise a dialog, and begin to actively converse with the author without interrupting the code editing... and everybody else with edit-rights should be able to see ongoing conversations so they can decide whether to join in. Automatically saves conversation to the code page's respective DiscussionPage, complete with signatures and times.
- Just because I have it reserved doesn't mean other people can't still edit it (and, e.g., fix a spelling mistake or missed parens); it is primarily a courtesy capability and a communications support capability.
Syntax highlighting: WebServer provides underlying divisions ('div' items with 'class' properties) indicating syntactic components (keywords, operators, top-level components). WebServer takes preference-specifications from the UserPage to provide a default CSS, but users can also utilize their own CSS to display these. The underlying divisions are not considered part of the source. In fact, purely display properties should rarely ever part of the source (since this Wiki is based on the semantics of the content). However, a language that provides arbitrary annotations capability could utilize such annotations to specify certain purely display annotations (whether or not they are discouraged). Essentially, the DOM/div components would contain annotation-information as values, identifying certain pieces of code as being 'keywords', 'operators', or even as evaluating to a known value at the time of last parse.
Potentially TiddlyWiki navigational style of 'linking' between words - i.e. multiple words can appear on one page (using AJAX) and can be edited in this form - especially for 'small' words (since this makes 'small' words a lot easier to use in practice). Of course, I'd still want the ability to easily edit from multiple tabs, so perhaps capturing something like MOD_ALT+onClick would be good for opening a local division/frame. This would also be nice for ProgressiveDisclosure.
If the page is in a 'bad' state that won't parse or typecheck, and is left that way for long enough, the WikiIde server should go ahead and let the editors know (via squiggly lines and hover-text, for example) where the code appears to be in a bad state.
Suggestions and CodeCompletion: This requires a server-side capability to take a cursor position and a page and send back a list of suggestions... and should happen based on user-prefs. This can be done with Ajax in much the same way as spelling-errors can be highlighted. Doing it efficiently will happen eventually, too, which requires only that the server itself keep track of apparent edit-sessions and index to support suggestions. Hover-text based on automatic annotations of code would also be useful and doable.
Checkpointing Pages: Simply because a page was the last edited does NOT mean it starts getting compiled into projects. Pages must be checkpointed by users (or robots). Checkpointing a page involves (albeit probably not strictly - i.e. use warnings, not restrictions):
- ensuring it parses,
- ensuring it typechecks (or at least doesn't have any obvious internal type-errors).
- any top-level assertions that don't depend on provisionals or aspected constructs pass their tests,
- running any other by-hand unit-tests or code-proofs one might wish to perform.
- inspecting the code (or at least the distance from the last trusted checkpoint) for any malign code injections.
- Potentially 'signing' the checkpoint with a PublicKey (stamp of approval) - most likely via use of a 'tag' property that gets dropped into a WikiIde database. This is the basis for a WebOfTrust, where a project can choose to use the most recent checkpoint given signature by someone it trusts, which was mentioned as a required feature (above) due to AssumeGoodFaithLimitations. The existence and support for signed and stamped checkpoints, along with the name spaces where necessary (for branching), should also reduce (though not eliminate entirely) the need to 'lock' pages or have bots maintaining them against the few random malicious arses that humanity has to offer...
- Potentially tagging the checkpoint with other properties or predicates such that you can easily find it later as part of a query.
Starting a 'Checkpoint' of a page sort of 'freezes' a view of the page and prepares it for a rapid edit-compile-test cycle to get it into a 'stable state', mostly clearing out silly logic errors that invariably occur (where "compile" really means "ensure it parses and typechecks") - what one might call a 'pre-checkpoint' state that is initially owned by the programmer who started the checkpoint.
- Because Checkpoints are also for security purposes, it must be possible to forbid other (particular) people from attempting to edit the pre-checkpoint page.
- However, to keep things convenient, it should be possible for other editors to join in on the edit, and thus it must be possible to have the original pre-checkpoint author to decide to allow others to join the edit (e.g. to see a dialog listing those interested in helping, and anyone with edit priveleges can let anyone else join).
- It should be possible to identify when somebody is in the act of checkpointing a page, thus to avoid accidental branching of the page. Perhaps 95% of the time, people will be willing to just let someone else finish their checkpoint before getting back to editing.
- It should be possible for programmers sick of waiting on the checkpoint to finish to branch the page and just start editing again. This makes the 'checkpoint' a sort of 'side'-branch to the main page. Fortunately, the 'checkpoint' (once marked as completed) is a 'fixed' entity that will no longer see edits, so it isn't a 'true' branch. Unfortunately, it does mean that any changes made during the pre-checkpoint cycle will need to be ported back to the 'main' branch by hand (a merge-tool would help).
  - True (editable) branching must be discouraged. Branching is extremely difficult to handle elegantly in a collaborative environment such as a WikiIde. In particular, problems arise when attempting to figure out to which branch a particular WikiWord should lead. Besides, branching is somewhat AntiSocial?, a violation of the WikiWay, and tends towards anti-integration of projects. (See ManaMana for a discussion of some of the problems, and as a demotivating example regarding the value of branching.)
  - I suggest that 'true' branching be limited to explicit efforts performed by, say, copying a particular version of a common page into a project's namespace then specifying to the linker (via the ProjectPage?) that the specified page is to be prefered whenever a particular WikiWord is to be linked. This has the SocialEngineering effect of discouraging branching, since it isn't the laziest thing, while avoiding the politics problems that come with inability to branch when necessary. There is, of course, still a large cost of 'splitting' or 'combining' a page, but that's where access to backlinks and dependencies-lists are important.
  - Most of the time the checkpoint would appear in the trunk, uninterrupted by any eager programmers (just like in WikiWiki, I anticipate that most of the time there is only one user interested in performing an edit on a given page, minus the tendency to pounce onto RecentChanges; in that case, the 'courtesy' notice that someone is running a checkpoint, plus the possibility of joining the checkpoint effort, should avoid most branching). In those cases of 'branching', what it might look like (in the history) is a simple 'checkpoint' followed by somebody rolling-back the edits and initiating new edits. In a sense, use of checkpoints creates 'macro-versions' as distinct from 'micro-versions'. These macro-versions are the ones used for code, while micro-versions are for editing between macro-versions.
Special tags and predicates available distinct from content for associating extra tags or predicates with a page. I.e. instead of dropping a 'Category' into a page as content, you simply 'attach' it to a page. These go directly to a WikiIde-wide propositional database (like DataLog) and are usable (along with the stuff -inside- the pages) in queries, providing a fundamental approach to complex reflection and semantic-web association within the WikiIde. Keeping tags separate from content makes it much easier to:
- tag 'fixed' pages including both particular or 'old' checkpoints or 'object'-pages that represent actual process and service objects.
- use tags to sign versions of a page
- create tags that associate two or more pages directly (PredicateOver(ThisPage,AndSomeOther) - appears in the 'tags' list of three pages: 'PredicateOver' (or possibly 'WikiTags:PredicateOver'), 'ThisPage', and 'AndSomeOther').
- Tag components of the Wiki as 'deprecated' or perhaps tainted even while they are in use
  - UserStory: If a patent violation or security risk is identified for a particular checkpoint of a page, it would be nice to mark it as 'tainted' then be able to ask, in a single query, for ALL 'tainted' project/objects currently running on the Wiki, plus for the e-mail address of the 'administrator' for that object/service, in order to deliver a mass e-mail regarding its ultimate shutdown or reset (e.g. 'at the end of the week, unless you take care of it first').
  - Access to this DataBase must be available as a service within the WikiIde so that the above UserStory can be implemented as a service within the WikiIde.
Ability to rapidly select the history page, see the recent checkpoint page, see the exported 'values', go to the 'edit'-version of the page (between checkpoints), go to a 'data' page to see all WikiIde-global explicit predicates & tags attached to the page (or a particular version thereof), quickly find ALL direct backlinks, quickly find ALL dependencies (everything a checkpoint of the page actually and potentially depends upon, each marked potential vs. actual, along with degree of indirection, possibly represented as a graph), and find ALL pages that depend upon or potentially depends upon on a particular checkpoint of a page, and degree of indirection, potentially represented as a graph)

Ajax and CSS are powerful enabling technologies for this sort of stuff, and I have no doubt that ALL of the above can be done - if not right away, then as the WebServer is upgraded from within the WikiIde (I'm aiming for a RuntimeUpgradeableCore). Actually, I bet that a good chunk of it could be done right away, and a good bit of that should be done right away, but I will not press for it. More advanced ObjectBrowser or plugin-based approaches would allow for greater efficiency and flexibility and IO-support, but requirement for them should be kept away from the basic WebServer product, and should be utilized either through particular services (e.g. when running a page as a CGI) or over special ports (since creating services inside the WikiIde to handle inputs on special IP ports should also be possible). (i.e. make users log into special ports or hook into particular services running ON the WikiIde to get advanced stuff).

I'm currently split on the idea of interjecting a component or two (potentially XsltLanguage) between the WebServer and the pages that has the general task of describing pages/URIs + a high-level 'purpose' as some sort of high-level InteractiveSceneGraph, such that the WebServer acts as a 'backend' to compile these high-level InteractiveSceneGraph back down into HTML & JavaScript & XMLHTTPRequest. One one hand: YouAintGonnaNeedIt (yet) - it would eventually allow for more flexible client-end support (especially regarding use of ObjectBrowsers). On the other hand, it might actually be simpler to inject two high-level transforms (with XsltLanguage or similar) than it is to write one low-level transform by hand, and coming up with this intermediate layer would offer a fine place to think about how to do it properly. I guess the choice will depend upon whether the SpikeSolution has a webserver that supports XML pipelining, and whether it seems the necessary output (which has JavaScript - but not necessarily very dynamic JavaScript) is easily done with the available tools.

I hereby claim dibs on the name "Wikid", even though at least two projects (bastards!) has already (mis)used the name for unrelated things. :) -- DaveVoorhis

Maybe WikiHeart?? I tossed it out as a name for the WikiIde open-port bindings page (in RuntimeUpgradeableCore), but it'd make a neat name for a WikiIde, and it would have the benefit of providing a clear project-name for updating the 'kernel' of the WikiIde from within the WikiIde (whereas 'wikid' sounds more like something you'd manipulate primarily from a unix command-line).

WikiHeart? is a bit too squishy and lovey for my taste, but maybe it will appeal to others. I can imagine the "I Wiki<3" bumper stickers and T-shirts. If it were left up me (thankfully, it isn't) I'd be more inclined to go with WiKernel?, or WiKore?, or something like that. -- DaveVoorhis

KingdomWikiHeart et al., eh? It is a bit Disney-esque. OpenIde? and WikiForge? would also be good (ForgeCore?).

I like WikiForge?. -- DaveVoorhis
- Willing to fight for the memetic space? WikiForge? is in use, but is not trademarked.
- In a word, no. I'd rather not get into a battle over naming, nor lose mindshare to divided attention. Something more unique, perhaps? -- DV
- We have a fair chance against our 'competion', I'd think (http://wikiforge.wikidot.com/). I doubt we'd have a battle over naming, and there'd be no divided mindshare regarding the use of Wiki as a platform for programming.
WikiWankaAndThePageFactory (or just WikiWanka for short) -- SamuelFalvo?
- Ugh. But WikiFactory? isn't bad (though it sounds like some sort of Wiki VonNeumannMachine?; OTOH, if we stick with the fully-bootstrapped approach - which I am not going to give up -, it would be fully capable of self-replicating). Focusing on alliterative forms: WikiWorkbench? is good (though I'm leaning towards WikiForge?)
- WikiFactory? appears to be well-used. -- DV
- Seems so.

I hope implementation won't have to wait until a particular is complete, lest this project be condemned to one of those infinite regresses where <x> can't be built until it can run on <yOS> which depends on <zLanguage> which will only run on <qPlatform>, which can't be written until <tSystem> is finished, and so on. (Or until it's proven that <tSystem> depends on <x>...) If it were me, I'd try to bodge together a quick-'n'-dirty minimal SpikeSolution out of existing tools, use it long enough to hate it (but learn from it), then throw it away and start over properly. I.e., BuildOneToThrowAway?. -- DaveVoorhis

Here's my ThinkingOutLoud (written in some haste) re bootstrapping a usable WikiIde:

There are four fundamental subsystems:

Core.
- This consists of, at minimum:
  - An HTTP server, consisting of:
    - The server binaries including executable file(s).
    - The server source code.
    - The server configuration files, configured to expose all source code in both subsystems, and to allow invocation of all scripts in all relevant subsystems.
    - Scripts to build, shut down, and restart the HTTP server.
  - Compilers and/or interpreters for all languages used in the system, including:
    - Source code to build the compiler or interpreter
    - Test scripts
    - Scripts to compile the source code to build the compiler or interpreter, and install the relevant binary files.
  - A Web-based text editor coupled with a versioning source control system (see below), configured to allow editing of any and all text files in the system, which works via the user's browser connected to the above HTTP server. Over time, this editor may be evolved by the users into the full WikiIde.
    - Source code and deployment scripts for the editor, as appropriate.
  - A versioning source control system, integrated with the Web-based text editor, such that any file saved by a user is recorded by the version source control system.
    - Source code to build the versioning source control system.
    - Scripts to compile the source code to build the versioning source control system, and install the relevant binary files.
  - A system of signed users to whom capabilities may be attributed. Not just anyone can be allowed to initiate a procedure that will send "rm -rf /" to the system.
    - Language and WikiIDE integrated support for easy use of capabilities; in particular, all actions should be performed with minimum capabilities, and users should be aware of which capabilities are going to be used when they execute a procedure or perform a build. Further, they should know which of their capabilities get 'embedded' (or 'signed') into services or procedures they create (and need tight control over it - ideally just those they add explicitly in the code AND give permission for in some dialog). This can't be made very annoying, of course, or we end up with something as bad as Vista - for the most part, procedures shouldn't require any special capabilities, or only use 'low security risk' capabilities.
    - We may wish to have a database of capabilities themselves, categorizing them by security risks, privacy risks, etc. This would help prevent annoying dialogs.
Primary heartbeat monitor.
- This consists of, at minimum:
  - A collection of test scripts that verify correct operation of:
    - The core.
    - The secondary heartbeat monitor. (See below)
- The primary heartbeat monitor shall continuously monitor the core and the secondary heartbeat monitor for new deployment of any of their components. When a change is deployed, the collection of test scripts will be executed. If any tests fail, the primary heartbeat monitor will roll back the changes (using the previous version of source control system, if it's what has just been modified) and reinstall the previous known-working version.
Secondary heartbeat monitor.
- This consists of, at minimum:
  - A collection of test scripts that verify correct operation of:
    - The primary heartbeat monitor.
- The secondary heartbeat monitor shall continuously monitor the primary heartbeat monitor for new deployment of any of its components. When a change is deployed, the collection of test scripts will be executed. If any tests fail, the secondary heartbeat monitor will roll back the changes and reinstall the previous known-working version.
User space.
- This consists of files that may be freely modified by users, and scripts that may be edited and executed by users. This provides the basis for constructing any and all applications and content unrelated to the core or the heartbeat monitors. No component of the core, or either heartbeat monitor, may be dependent on any component living in user space. However, components developed in user space may be copied to the core or either heartbeat monitor.

The fundamental constraints are as follows:

When a change of the core is deployed, no changes may be made to the heartbeat monitors until the primary heartbeat monitor has validated (and possibly rolled back) the core.
When a change of the primary heartbeat monitor is deployed, no changes may be made to any subsystem until the secondary heartbeat monitor has validated (and possibly rolled back) the primary heartbeat monitor.
When a change of the secondary heartbeat monitor is deployed, no changes may be made to any subsystem until the primary heartbeat monitor has validated (and possibly rolled back) the secondary heartbeat monitor.
The compilers/interpreters used to implement the heartbeat monitors, and their associated scripts, are as changeable as the rest of the system. However, deployment of new versions of these must be deferred until associated automated tests have verified that their deployment will not break the core, the heartbeat monitors, or the source control mechanisms.
The source control system is as changeable as the rest of the system. However, deployment of a new version must be deferred until appropriate automated tests have verified that its deployment does not break the source control mechanisms.

This is very back-of-napkin at the moment; I've been writing it as I conceive it, so it may be fundamentally flawed. I worry, in particular, that there may be dependency chains where breakage in a single rung may bring down the system. However, I think it should provide an (essential) independence between the languages used to implement the system and the system itself, and permit changing these as required. It also means it can be deployed with a minimum of development and can use existing tools - such as an Apache HTTP server and its source code, the Subversion revision control system and its source code, Perl/Python/PHP/Ruby/C/C++/etc. compilers/interpreters and associated source code, and any standard OS capable of running these.

-- DaveVoorhis

I really like the use of heartbeat monitors - a system with two hearts where each can fix the other and one can fix the system is a fairly good basis for stability. I'll add them to the approaches listed in RuntimeUpgradeableCore (to be used in combination with, not instead of). Obviously you're depending on an external scheduler to support the heartbeat monitors, but (for the SpikeSolution) I think that choice is appropriate; in the long run, I want the primary scheduler to be part of the WikiIde, which is necessary so it can be upgraded from within the WikiIde to support distributed operations and server-farms and bare-metal installation. This, of course, means that the heartbeat monitors will almost certainly be running on an upgradeable scheduler. Fun, eh? The scheduler + network reachibility (either of which is useless without the other) will ultimately be primary points-of-failure in the WikiIde system (whether it is distributed or not), which really means that we'll want to eventually support the construction of rescue-disks.

On the User Space issue, I believe that proper Project and ProjectPage control would be sufficient to the task. In particular, the Heartbeat and Secondary Heartbeat and Kernel projects can 'depend' on these files at build-time, but the pages utilized by the project shall be selected at a particular version-identifier - aka. a logical 'freeze'. All projects, not just the forge service project and its hearts, shall have this ability to (logically) freeze pages so they aren't upgraded whenever said page is modified. In a sense, I really want all projects to be first-class entities, on exactly the same logical level as the WikiIde core services (and even capable of adding to the open IP ports, albeit only through a particular (potentially capability-secured) page).

I'm not convinced of the value of attempting to support multiple languages; the ability to properly and meaningfully link and integrate and modify service (typically just the content of the page being served), not the ability to compile and run services, is what the 'Wiki' aspect is all about. The use of the OS executive environment creates a very fragile and largely unintegrated and poorly linked system that I fear will fall apart quite easily, and it will also have the failings for multiple-languages that I described in ProgrammingInWiki. And if the OS is used for some 'special' things and not for everything else, then the WikiIde really isn't a first-class service of the WikiIde.

Consider if our WikiIde did support upgrading Apache. This means supporting Makefiles, C++, a Filesystem filled with C++ '.h', '.hpp', '.cpp' pages (of which only the '.h' pages could essentially be referenced from other pages) possibly automake, and lots and lots of directories. I won't even consider trying to make the C++ language & compiler itself upgradeable - that doesn't get pretty. The 'WikiWay' would involve describing services or algorithms or values or data on pages identified by WikiWords on, as much as possible, a 'flat' namespace (excepting where projects need special implementations, common page-addendums like discussions, user:prefs, etc.). This avoids any need to:specify:words:like:this or worse (namespaces at least can be associated with 'using namespace' declarations and autogen disambiguation lists). In the WikiWay, these external algorithms, values, and data are automatically integrated into the local project where the WikiWord names are used, which requires using a language that knows how to do the integration (i.e. a common language, or a 'super'-language that is really smart about it). In addition to providing an example of what not to do (and one taking tons and tons of pages), the Apache implementation would also divide mindshare when it comes to fixing perceived 'webserver' problems: one can go about attempting to fix Apache or some component of it, or one can start building a WebServer that really is a FirstClass WikiIde product.

We may be better off with having the WebServer and such be considered 'external' and SecondClass system until such a time as we are able to create a project within the WikiIde that can allow for porting of the service. Thus, instead of attempting to treat the webserver and scripts and heartbear monitors as some sort of FirstClass WikiIde project from the beginning, we simply use it as a scaffolding; and, while being supported by the scaffolding, we build the service to our own vision (so that later we can upgrade it from within the Wiki), run a bunch of tests, then carefully (by hand, and with a system backup available should we fail) remove the scaffolding and henceforth utilize the FirstClass implementation. and the various other services. I.e. we essentially bootstrap every single service by hand. And the core component we need to be there, from the beginning, is the language. (This is essentially the roadmap I possessed.)

Not that I'm dissing your design or the constraints, and I certainly like the heartbeat monitors. But, keeping in mind the bigger picture of the WikiIde goals, I foresee the attempt described above (even if you intend it be there only for the SpikeSolution) to be ultimately harmful to my goals with the WikiIde. I also fear that it will become almost impossible, or at least a significant battle, to ever wean the WikiIde off these early influences. (Or perhaps I misinterpreted you when you said that, say, the web server could be upgraded just like any other part of the system? Let me know if it appears I've misunderstood your intentions.)

Security Concerns for SpikeSolution

We can't really afford to expose 'system("rm -rf /")' (or any equivalent thereof) through our WikiIde SpikeSolution... not without having to password protect everything and be extremely careful about who we allow to play in the sand. I think that we'll need to have a security solution built into our SpikeSolution before we open anything to the public.

At the very least, we need to have signed users to whom we can then assign capabilities. It's somewhat difficult to give a capability to a user when they don't have an authorized handle.

I've addended the above to adjust for this.

Today, these first and second heartbeed monitors are named ContinuousIntegration.

No. ContinuousIntegration pre-dates the notion of using heartbeat monitors. Heartbeat monitors, as described above, are a specific technical architecture that prevents WikiIde authors from irretrievably breaking a live WikiIde via erroneous modifications to its own core code. ContinuousIntegration is a broad vision of development, at best a general strategy -- it is not a specific architecture.

It would be more accurate to say that the heartbeat monitors are technical infrastructure that makes it safe to perform ContinuousIntegration on a WikiIde's kernel via the WikiIde itself.

I've initiated a project http://keinze.mbwiki.de/pmwiki.php?n=Main.Mbwiki -- MirkoBlueming 21.05.2012

Anything happening here recently? Just wondering whether people have noticed BespinEditor, which seems a good candidate to build a WikiDE.

Contributors: DaveVoorhis

MarchZeroEight

SeeAlso: QedWiki, ProgrammingInWiki, WikiOs, OsWiki, ReflectiveWiki, WikiWithProgrammableContent, BespinEditor

CategorySoftwareTool