Issue tracker, what I need, how I need it

I need an issue tracker to track issues.

Issues are things like:

• Todo lists
  
• Things which happened and I want to archive
  
• Bug tracking and resolving
  
• Wish lists
  
• Blog
  
• Content management
  

This all are issues. Managing content like in a CMS is an issue, because content usually arises because of some issue, like an accident, I want to tell something, I have an idea, whatever. So even blogging is just another form of an issue.

Must keep any content

So if I tell of an issue tracker, this means, I must be able to store any data into it:

• Posts
  
• Messages
  
• Mails
  
• Files
  
• Graphics
  
• one-liners
  

Must keep a history but must not be a history

A good issue tracker also is a Wiki with an edit history. But it is not the history. So all the issue trackers which focus to the history of an entry are all wrong.

Example:

• Some user has a problem with one of my software. He writes me an eMail. I forward this eMail to my issue tracker. The eMail contains some screenshot.
• I now work on this issue. The software is fixed. The fix is uploaded. Perhaps the fix just is a part in the documentation of the software. But nobody reads documentation, right?
• I now make this information public. This is, I publish it somewhere on my homepage. So the issue tracker must keep this homepage, too. There is no way else to bing that information easily to my homepage, right?
• The published information must not be the history list how the issue evolved. This is bullshit. Nearly 80% of the web is bullshit like this. You have a problem. You google. You find somthing. You have to scroll 1 MB or more of all those old forum posts to find just the single post which resolves your issue. Wasted time.
• So the published information must go where it is expected. Perhaps it must add to the FAQ section of my program on the homepage about this software. So the issue must get added a QUESTION and the ANSWER. Probably the QUESTION is made up the screenshot which the user sent in, the ANSWER is the last fix (a part of the eMail sent to the user).
• So the VIEW to the ISSUE is completely different from what you usually see on issue trackers.

Must have a workflow

When the issue is forwarded to the issue tracker, the User who sent the eMail wants to be informed. So the issue AUTOMATICALLY must get the User as the originator. But originator is bad, better is a watch list, so the user must automatically placed on the watch list.

When I edit the issue, in 90% of the time I just add some note which is cryptic information just for me, not for the user. So the user does not want to get such information. I do not want to write information such, that the user understands it. But if I have resolved the issue, I will write something the user wants to get, so this must go to the user.

However there is more to it. The user perhaps wants to be informed about how far I got to fix the problem. So the user must be able to read my log. Perhaps with some things hidden, because these must not be disclosed to a third party (perhaps I note a password just to have it handy). So the user must get an eMail telling that the issue now is processed, and in which state it is. This must be automatic, such that I do not need to do it, I do not want to do it, I do not need to remember doing it.

And there is even more to it. Perhaps I get stuck and need more information from the user. Now I write "well, your screenshot does not tell it, please gimme more info" and change the status to something like "need more info". In this case the issue automagically must be disappear at my side and appear at the user's side for processing.

But there is more to it. I do not want to loose contact to the issue. So after some days of inactivity or whatever it must re-appear at my side. All without me thinking of it, just automatic. It must not be my task to do all this surveillance, this all must be done by the issue tracker.

But there is more to it. For example if there are more than one watcher, then these watcher usually cannot give me this info. So it is wrong to notify the watcher. The originator/creator must be notified. But the creator can change. So there must be a notifier list besides a watcher list. Because perahps the user says, that his brother can tell me more, so we now have 2 notifiers.

Must be easy to add

Adding an issue must be easy. Easy means:

Only as few fields as needed to fill out. More things to fill out later.

What are the few fields?

• The eMail-Address, pre-filled if you are logged in.
• The issue, just a text field
  
• An option to upload a file or screenshot
  
• The submit button
  

But there is more to it.

• eMail address is optional. So you do not need to give an eMail address.
• Issue needs not to be filled out if you upload a file.
• File needs not to be uploaded if you filled the issue.
• There is no Captcha.
  

WTF? You read it.

There are two things not noted.

• If you leave out the eMail address, a Captcha with a secret code is shown. This code must be entered to finish the post.
• If you enter an eMail-address which is unknown, the eMail address is verified. This means the secret code is sent to the eMail address as well as an easy-clickable URL. However the Captcha still is shown with the same secret code.
• If the issue is confirmed with the secret code (or the URL), the issue is forwarded to processing. Else it is kept back and NOT forwarded.
• If the eMail-address is confirmed and new, you get the option to create an account using this eMail address. This account then can be associated with your OpenID based on the eMail address.
• If the eMail-address is known already, and OpenID is supported by this eMail address, this automatically logs you in, if this is allowed. So you do not need to remember passwords.
• If auto-login is not possible, you must enter your password. However you still can skip that step and continue the other way (Captcha / verify).

Why this all? It's easy.

• If you are blind and cannot read Captchas you need another way. This is provided by eMail verify.
• If you are in an insecure environment and do NOT want to enter your password and cannot use OpenID, you still can enter an issue.
• If you do not have an eMail address or want to do it anonymously, you just write down the secure code, and you can access the issue later on.

How is the secure code made up?

Here is my idea, if you have a better idea go for it.

• The secure code is of 5 random digits. Each digit has 32 possible characters. This makes 25 bit information.
• The secure code is prefixed by N digits and a dash. Both are pre-entered for you.
• The secure code is suffixed by a dash and a 1 digit check.

So the secure code might look like:

NNNNN-SSSSS-C

To recover your issue you need to know NNNNN-SSSSS-C. To send the issue you need to enter SSSSS which is shown in the captcha.

If you lost NNNNN and C this is not a big problem. If you know some words about the issue you can search for it. NNNNN then is shown to you. C can be left away, if there is enough evidence about the issue. If there is an eMail address associated with the issue, you can even recover SSSSS.

The issue internally is handled as NNNNN. Note that NNNNN also is a number. NNNNNN allows 16 million issues. So NNNNN has a dynamic range. As issues are biased at 100000 the minimal length of NNNNN is NNNN.

Why issues are biased?

There are several biases in this system. The first bias is the issue bias. It defaults to 100000. This means there are 100000 documents below the first issue which can get assinged a number.

If you like you can customize the bias. You can set it to 0 as well.

To access an issue, you can either enter NNNNN or you can enter a number with a bias. There is a default bias (based on the context) or you can give a bias-prefix and a signed number.

Example:

• :1 is the absolute document number. The bias for : is 0.
• -1 is the current bias less 1. So it is a relative number.
• a1 is the document 1 with bias a. Bias a can be set to any value.

Now we have some ambiguity:

a1 can either be the document A1 or document 1 biased by a. Note that lower case letter. If entered as uppercase, it is the document, if entered as lowercase it is a bias.

Several letters / characters are problematic:

• 0 O o Q ()
  
• 1 l I 7 / \ ! | ' ` ( )
  
• 2 Z z
  
• 4 H
  
• 5 S s $ ?
  
• 6 G b
  
• 8 B &
  
• 9 g q
  
• C c ( < [ {
  
• P p D R
  
• v V u U \/
  
• m rn
  
• X Y x y ><
  
• T t +
  
• . ,
  
• _
• = :
  
• ) > ] }
  

It has to do with lazy writing if you are in a hurry or smear effects of your pencil after it was put in your pocket and perhaps got wet by sweat. So these letter combinations (and perhaps some other too) must be used such that they cannot be mixed accidentally (you can only take one of the letters).

This leaves us with following alphabeth of 32 easy to distinguish characters:

• The code is case insensitive
  
• If you use wrong characters, it will notify you.
  
• Users have to use the right caps, developers have to enter all uppercase.
• Note that only the number 3 survived.
  

That's it about codes.

BIASes are used to shorten several things. Like access page 27 as p27 is more easy than to enter the ID 293483 of documents based at 293457 (p1 is 293457 as p0 is unnatural to human beings).

You get the idea.

Must be easy to edit

When I want to change anything, I must be able to do so.

There must not be clutter. So information I do not want to see must not be present.

This also involves configuration. If I need to add 2000 users, I do not want to see a form. If I want to add 1 user, I want to see a form.

Must keep a history on everything

Any change must be logged. So there is no normal logging, there always is some log which is kept. Even for the "register" thing, there must be kept a log.

The log must be able to play forward and backward. So it always is "before+after" (a bi-directional diff).

Must be integrating

There must be some interfaces to the system. The best idea I have so far is a generic JSON interface to the backend. Then frontends attach to this backend.

The frontends can be PHP (for web based Lynx) or JavaScript (for modern AJAX browsers) or standalone (Adobe AIR, whatever). Also there must be some middleware (eMail gateway incoming and outgoing).

So the backend probably is best implemented using Python with just a JSON interface. The rest is implemented using some frontends. There can be some Python default frontend, but I think best is to change the language, such that to have a barrier between the core and the frontend. This helps to reduce errors (like SQL injection, XSS etc.).

Must support rich content

For normal users just raw text is ok. But many users send HTML based eMail. The system should not loose this content.

However it must also place a barrier. So external images must not be in the message. Images must be pulled into the content before they are presented.

Also some users perhaps want to edit it as rich texts. So you take a document, edit it right in the browser and place it back.

Perhaps you only want to edit parts of a document. Such that you cannot harm other sections. This all must be supported by the backend.

Rendering shall be WYSIWYG. This is, while you edit it in the rich editor the preview is shown. This preview then is saved along with the raw code. So rendering is not done on the frontend, it is done while editing. However the rendering can be change in case you re-render it. But this only is a view. So views can be pre-rendered or on-demand.

Not anonymous, but public and published

The default setting is to not publish anything. You can choose which version to show publicly or to other people.

So you can have one public version, one for "normal" people after they are logged into and another for the originator, several for several editors and one complete for some admin. Perhaps there even is no complete view to anybody not even the admin.

Crypted documents

Documents shall be stored encrypted and shall be decrypted on the fly. The decryption happens in the front end. Only the published parts are stored unencrypted.

Why?

Because I, the developer, perhaps want to note my password in the history. But I do not want to share that password with anybody else. Perhaps I want to share it with 2 certain people, but changing the permission on the global document scope must not publish my password accidentally.

So the default is to create crytped content, and to decrypt it only if requested. To not accidentally remove the cryption on certain parts, encryption must be lockable, this is, it cannot be enabled again, the content must be re-published (edited) to change that flag.

So the transfer of crypted data between backend and frontend (which can happen on an insecure channel such as HTTP) still is crypted, as the frontend must decrpyt everything.

Adaptive interface

The interface must be very general. The idea is that there are no predefined URLs, such that you can attach any data into the system. So you create a document and map a view to it into the URL space somewhere.

This way you can replace some old system with this new one easily - just import all documents and attach the old URL to them.

Different views to different information

The information which belongs together must be viewable together. But it also must be able to view separately.

For example, you can refer to some attachment in one issue from another, such that both share the same attachment.

Semantic search

As the different information is separated easily you can search for it easily, too. This must be supported.

Extensible

Extensions shall be easy to add. This can be accompanied by the system managing it's own code base.

That's it, basically

With this here in place, you usually can create whatever you want:

• CMS
  
• Wiki
  
• Blog
  
• Issue Tracker
  
• Webmailer
  
• VCS
  
• Document Management
  

Actually, all this noted above is all the same. It's just another view to the same things.

Future

By supporting WSDL/XML/SOAP the system could even easily integrate into any other framework capable of provinding this interface.

What it is not

It is not a queuing system or a middleware (SOA). So it is not able to handle millions of records efficiently, as it is more a document management for big huge documents instead of many small information pieces.

-Tino, 2010-11-09