view Side-By-Side changes
Document:draft-melnikov-imap-condstore-02.txtdraft-melnikov-imap-condstore-03.txt S. Hole Expires:January 2001March 2002 ACI WorldWide/MessagingDirectJulySeptember 2001 IMAP Extension for Conditional STORE operation Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet- Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (C) The Internet Society 2001. All Rights Reserved. 0.1. Open issues 1). Should conditional STORE be atomic accross message set (i.e. either all messages in message set weren't changed since and condtiontal STORE succeeds or operation fails for all messages)? This can be very expensive for some servers. 2). How specify different UNCHANGESINCE for different flags in the same STORE? Do we want such granularity anyway?2).3). The document assumes that each flag has a corresponding ANNOTATE entry. This has to be synchronized with ANNOTATE draft.3). "If the "modtime" of any metadata item specified in STORE operation4). Add support forany message in the message set is greater than the unchangedsince value, then the store fails with a MODIFIEDSORT extension? 0.2. Change History Changes from -02 to -03: 1. Changed MODTIME untagged response to MODTIME response code. 2. Added MODTIME response codethat includes message set of all messages that failed UNCHANGESINCE test." This basically means that server must continue conditional store operationtofind all messages for which test fails. Is it reasonable, or shouldthetext be changed from "all messages" to "at least one message". 4). MODTIME Untagged Response specifiestagged OK response for SEARCH. Updated examples accordingly. 3. Changed rule for sending untagged FETCH response as amessage set modtime applies to. Can anybody thinkresult ofa reason why itSTORE when .SILENT prefix isa bad idea? Message setused. If .SILENT prefix isnot really required, because MODTIME untagged response must follow correspondingused, server doesn't have to send untagged FETCHresponses. Should we use UID set instead? 5). Should SEARCH MODTIME requireresponse, because MODTIMEuntagged response? SEARCH doesn't cause updates, unless they are the result of some other operation. However SEARCHresponse code already contains modtime. 4. Renamed MODTIMEforces servertoread modtime, so server can report this information anyway. 6). Add support for SORT extension? 0.2. Change HistoryMODSEQ to make sure there is no confusion between mod-sequence and ACAP modtime. 5. Minor ABNF changes. 6. Minor language corrections. Changes from -01 to -02: 1. Added MODTIME data item to STATUS command. 2. Added OK untagged response to SELECT/EXAMINE. 3. Clarified that MODIFIED response code contains list of UIDs for conditional UID STORE and message set for STORE. 4. Added per-message modtime. 5. Added PERFLAGMODTIME capability. 6. Fixed several bugs in examples. 7. Added more comments to ABNF. Changes from -00 to -01: 1. Refreshed the list of Open Issues. 2. Changed "attr-name" to "entry-name", because modtime applies to entry, not attribute. 3. Added MODTIME untagged response. 4. Cleaned up ABNF. 5. Added "Acknowledgments" section. 6. Fixed some spelling mistakes. Table of Contents <<To be completed later>> 1. Abstract There is a class of applications that wish to have multiple IMAP clients coordinate activities with a single shared mailbox. These applications need some mechanism to synchronize state changes for messages within the mailbox. They must be able to guaranty that only one client can change message state (e.g. message flags or annotations) at anypoint intime. An example of such an application is use of an IMAP mailbox as a message queue with multiple dequeueing clients. The Conditional Store facility provides a protected update mechanism for message state information that can detect and resolve conflicts between multiple writers. 2. Conventions Used in This Document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [KEYWORDS]. Formal syntax is defined using ABNF [ABNF] as modified by [IMAP4]. In examples, "C:" and "S:" indicate lines sent by the client and server respectively. The term metadata or metadata item will be used throughout this document. It references any system or user defined flag or an annotation [ANNOTATION]. 3. Introduction and Overview The Conditional STORE extension is present in any IMAP4 implementation which returns "CONDSTORE" as one of the supported capabilities in the CAPABILITY command response. Every read-write metadata of an IMAP message has an associated unsigned 64-bit value called modificationtimestamp (modtime).sequence (mod-sequence). This is an opaque value updated by the server whenever metadata item is modified. The value is intended to be used only for comparisons within aserver, not as an accurate timestamp.server. However the server MUST guaranty that each STORE command (including simultaneous stores from different connections on different attributes) will use differentmodtimemod-sequence values. Also, for any two successfull consequent conditional store operations performed in the same session,modtimemod-sequence of the second MUST be greater thanmodtimemod-sequence of the first.Modtime is described more fully in section 3.1.1Note, that the latter rule disallows the use of[ACAP]. Modtimesystem clock as mod-sequence, because if system time changes (e.g. NTP client adjusting time), next generated value can be less than previous. Mod-sequence allows the client that supports CONDSTORE extension to track whether the value of particular flagwashas changed since somemoment in time.known moment. Whenever the state of a flag change (i.e. the flag is added and before it wasn't set or the flags is removed and before it was set) the value of modificationtimestampsequence for that flag MUST be updated. Adding the flag when it is already present of removing when it is not SHOULD NOT changemodtime.mod-sequence. Change to any flag MUST also update per message mod-sequence. Each flag SHOULD have separatemodtime,mod-sequence, for example change to \Draft flag SHOULD NOT affectmodtimemod-sequence for \Deleted flag. Server that supports per flagmodtimes (i.e.per message mod-sequences (i.e. satisfies the latter SHOULD) MUST also report"PERFLAGMODTIME""PERFLAGMODSEQ" in CAPABILITY command response. When message is appended to mailbox (via APPEND command or using external mechanism) the server assigns the current servertimestampmodification sequence to every flag or annotation specified in the APPEND command. When an annotation is removedmodtimemod-sequence SHOULD be preserved. This extension makes the following changes to the IMAP4 protocol: a) extends syntax of the STORE command for allowing to specify STORE modifiers b) adds MODIFIED response code that should be used with NO response to STORE c) adds a newMODTIMEMODSEQ message data item for use in the FETCH command d) adds a newMODTIMEMODSEQ message data item for use in the SEARCH command e) adds a newMODTIME untagged ResponseMODSEQ response code f) adds a new OK untaggedresponsesresponse for SELECT and EXAMINE g) addsMODTIMEHIGHESTMODSEQ status data item The rest of this document describes protocol changes more rigorously. 4. IMAP Protocol Changes 4.1. OK untaggedresponsesresponse for SELECT and EXAMINE This documentaddadds a new OK untaggedresponsesresponse for SELECT and EXAMINE commands. A server supporting CONDSTORE extension MUST send the following OK untagged response with any successful SELECT or EXAMINE command. OK[HIGHESTMODTIME <modtime-value>][HIGHESTMODSEQ <mod-sequence-value>] The highestmodtimemod-sequence value for any metadata item of any message in the mailbox. Disconnected client can use the value ofHIGHESTMODTIMEHIGHESTMODSEQ to check if it has to refetch flags from the server. If the value stored in clients cache is less than the value returned by the server, this means that some metadata items on the server changed since last synchronization and client has to update its cache. Client MAY use SEARCHMODTIMEMODSEQ as described in 4.4 to find out exactly what metadata items has changed. Example: C: A142 SELECT INBOX S: * 172 EXISTS S: * 1 RECENT S: * OK [UNSEEN 12] Message 12 is first unseen S: * OK [UIDVALIDITY 3857529045] UIDs valid S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited S: * OK[HIGHESTMODTIME "20010715194045007"][HIGHESTMODSEQ 20010715194045007] S: A142 OK [READ-WRITE] SELECT completed 4.2. STORE and UID STORE Commands Arguments: message set OPTIONAL store modifiers message data item name value for message data item Responses: untagged responses:FETCH, MODTIMEFETCH OK untagged responses: MODSEQ (See section 4.5) Result: OK - store completed NO - store error: can't store that data BAD - command unknown or arguments invalid This document extends syntax of the STORE (and UID STORE respectively) command (see section 6.4.6 of [IMAP]) to include optional STORE modifiers. The document defines the following modifier: UNCHANGEDSINCE If the"modtime"mod-sequence of any metadata item specified in STORE operation for any message in the message set is greater than the unchangedsince value, then the store fails with a MODIFIED response code that includes message set (for STORE) or set of UIDs (for UID STORE) of all messages that failed UNCHANGESINCE test. Example: C: a101 STORE 7,5,9 (UNCHANGEDSINCE"20000320162338")20000320162338) +FLAGS.SILENT (\Deleted) S: a101 NO [MODIFIED 7,9] Conditional STORE failed In spite of failure of conditional STORE operation for message 7 server continues to perform conditional STORE to find all messages for which operation will fail. Use of UNCHANGEDSINCE with atimemodification sequence of"00000101000000"0 will always fail if the metadata item exists. Example: C: a102 STORE 12 (UNCHANGEDSINCE"00000101000000")0) +FLAGS.SILENT ($MDNSent) S: a102 NO [MODIFIED 12] Conditional STORE failed If operation is successful the server MUST update the"modtime"mod-sequence attribute for every metadata item that was changed. Untagged FETCH response MUST be senteven if(unless .SILENT isspecifiedspecified) and it MUST includeMODTIMEMODSEQ message data item as described in 4.3. Also server MUST senduntagged MODTIMEMODSEQ response code to indicate that client has received all updates to metadata items which havemodtimemod-sequence values less than or equal to the indicatedmodtimemod-sequence value. Example: C: a103 UID STORE 6,4,8 (UNCHANGEDSINCE"200012121230045")200012121230045) +FLAGS.SILENT (\Deleted) S: * 1 FETCH (UID 4MODTIME ("200012121231000"MODSEQ (200012121231000 "/message/flags/system/\\Deleted""200012121231000"))200012121231000)) S: * 2 FETCH (UID 6MODTIME ("200012101230852"MODSEQ (200012101230852 "/message/flags/system/\\Deleted""200012101230852"))200012101230852)) S: * 4 FETCH (UID 8MODTIME ("200012121130956"MODSEQ (200012121130956 "/message/flags/system/\\Deleted""200012121130956")) S: * MODTIME 1:4 "200012121231000"200012121130956)) S: a103 OK [MODSEQ 4,6,8 200012121231000] Store completed Example: C: a104 STORE * (UNCHANGEDSINCE"200012121230045")200012121230045) +FLAGS.SILENT (\Deleted $Processed) S: * 50 FETCH(MODTIME ("200012111230045"(MODSEQ (200012111230045 "/message/flags/system/\\Deleted""200012111230045"200012111230045 "/message/flags/system/$Processed""200012111230045")) S: * MODTIME 50 "200012111230045"200012111230045)) S: a104 OK [MODSEQ 50 200012111230045] Store completed In the latter example UNCHANGEDSINCE value is checked againstmodtimesmod-sequences for both flags. Note: If the message is specified multiple times in the message set and the server doesn't internally eliminate duplicates from the message set it MUST NOT fail conditional STORE operation for the second occurrence of the message in the message set if operation completed successfully for the first occurrence. For example, if the client specifies: a100 STORE 7,3:9 (UNCHANGEDSINCE"200012121230045")200012121230045) +FLAGS.SILENT (\Deleted) the server must not fail operation for the message 7 as a part of processing "3:9" if it succeeded when the message 7 was processed the first time. 4.3MODTIMEMODSEQ message data item in FETCH Command This extension adds anMODTIMEMODSEQ message data item to the FETCH command. This allows clients to retrievemodtimemod-sequence for various metadata items for a range of messages in the currently selected mailbox. Syntax:MODTIMEMODSEQ <entry-names> TheMODTIMEMODSEQ message data item, when used by the client in the FETCH command, takes a list of metadata items. For a flag <flagname> the corresponding entry-name has a form "/message/flags/system/<flagname>". Empty list requests server to return only per-messagemodtime. MODTIMEmod-sequence. MODSEQ message data item causes server to returnMODTIMEMODSEQ fetch response data item. Syntax:MODTIMEMODSEQ (<permessage-modtime><permsg-modsequence> <entry-name><modtime-value><mod-sequence-value> ... )MODTIMEMODSEQ response data item contains per-messagemodtimemod-sequence and possible empty list of requested metadata items and their correspondingmodtimes.mod-sequences. Example: C: a FETCH 1(MODTIME(MODSEQ ("/message/comment" "/message/flags/system/$MDNSent")) S: * 1 FETCH(MODTIME ("20000624140000"(MODSEQ (20000624140000 "/message/comment""20000624140000"20000624140000 "/message/flags/system/$MDNSent""20000624140000"))20000624140000)) S: a OK Fetch complete 4.4MODTIMEMODSEQ criterion in SEARCH TheMODTIMEMODSEQ criterion for the SEARCH command allows a client to search for the specifiedmodtimemod-sequence of a metadata item in a message. Syntax:MODTIMEMODSEQ <entry-name><modtime-value><mod-sequence-value> Messages that have modification counter for metadata item <entry-name> with value equal or greater than<modtime-value>.<mod-sequence-value>. This allows a client, for example, to find out which messages contain metadata items that have changed since the last time it updated its disconnected cache.Examples:If client specifies MODSEQ creterion in SEARCH command and server returns non empty SEARCH result, server MUST also return MODSEQ response code in the tagged OK response for all messages returned in untagged SEARCH. See also 4.5. Example: C: a SEARCHMODTIMEMODSEQ "/message/flags/system/\\draft""20010320162338"20010320162338 ANNOTATION "/message/comment" "value" "IMAP4" S: * SEARCH 235 6 7 1113 1712 18 19 20 23 S: a OK [MODSEQ 2,5:7,11:12,18:20,23 20010917162338] Search complete In the above example, the message numbers of any messages containing the string "IMAP4" in the "value" attribute of the "/message/comment" entry and havingmodtime "20010320162338"mod-sequence 20010320162338 for flag \Draft are returned in the search results. Example: C: a SEARCH OR NOT MODSEQ "/message/flags/system/$MDNSent" 20010320162338 LARGER 50000 S: * SEARCH S: a OK Search complete, nothing found 4.5MODTIME UntaggedMODSEQ Response code for successfulFETCH andFETCH, STORE and SEARCH Data: message setmodtimemod-sequence value MODSEQ response code can be sent in the following three cases: 1) Successful STORE UNCHANGEDSINCE results inuntagged MODTIMEMODSEQ response code being sent in tagged OK response to the client to inform of the latestmodtimemod-sequence of all metadata items specified in the STORE command.It would alsoMODSEQ response code contains message set mod-sequence applies to if is caused by STORE command or UID set it is caused by UID STORE. Example: C: a103 STORE 9 (UNCHANGEDSINCE 200012121230045) +FLAGS.SILENT ($Forwarded) S: * 9 FETCH (MODSEQ (200012121231000 "/message/flags/system/$Forwarded" 200012121130046)) S: a103 OK [MODSEQ 9 200012121231000] Store completed 2) MODSEQ response code in untagged OK response MUST beissuedsent by the server following a group of one or more unsolicited FETCH responses to indicate that the client has received all updates to metadata items which havemodtimemod-sequence values less than or equal to the indicatedmodtimemod-sequence value. MODSEQ response code terminating a group of unsolicited FETCH responses always contain message set mod-sequence applies to. Example: C: a103STORE 9 (UNCHANGEDSINCE "200012121230045") +FLAGS.SILENT ($Forwarded)UID FETCH 20138:* (FLAGS INTERNALDATE) S: *9101 FETCH(MODTIME ("200012121231000" "/message/flags/system/$Forwarded" "200012121130046"))(UID 20140 FLAGS (\Seen \Answered $Forwarded) INTERNALDATE "28-Oct-2001 15:44:25 +0300") S: *MODTIME 9 "200012121231000"12 FETCH (FLAGS (\Flagged) MODSEQ (200109121231000 "/message/flags/system/\\Flagged" 200109121231000)) S: * 16 FETCH (FLAGS (\Answered \Flagged) MODSEQ (200109121231000 "/message/flags/system/\\Flagged" 200109121231000 "/message/flags/system/\\Answered" 200109121231000)) S: * OK [MODSEQ 12,16 200109121231000] S: a103 OK Store completed 3) If client specifies MODSEQ creterion in SEARCH command and server returns non empty SEARCH result, server MUST also return MODSEQ response code in the tagged OK response for all messages returned in untagged SEARCH response. MODSEQ response code contains message set mod-sequence applies to if is caused by SEARCH command or UID set it is caused by UID SEARCH. 4.6MODTIMEHIGHESTMODSEQ status data items This document defines a new status data item:HIGHESTMODTIMEHIGHESTMODSEQ The highestmodtimemod-sequence value for any metadata item of any message in the mailbox. This is the same value that is returned by server inHIGHESTMODTIMEHIGHESTMODSEQ response code in Ok untagged response (See 4.1). Example: C: A042 STATUS blurdybloop (UIDNEXT MESSAGESHIGHESTMODTIME)HIGHESTMODSEQ) S: * STATUS blurdybloop (MESSAGES 231 UIDNEXT 44292HIGHESTMODTIME "200201011231777")HIGHESTMODSEQ 200201011231777) S: A042 OK STATUS completed 5. Formal Syntax The following syntax specification uses the Augmented Backus-Naur Form (ABNF) notation as specified in [ABNF]. Non-terminals referenced but not defined below are as defined by [IMAP4]. Except as noted otherwise, all alphabetic characters are case- insensitive. The use of upper or lower case characters to define token strings is for editorial clarity only. Implementations MUST accept these strings in a case-insensitive fashion. capability =/ "CONDSTORE" /"PERFLAGMODTIME" resp_text_code =/ "HIGHESTMODTIME" SP modtime-value"PERFLAGMODSEQ" mailbox-data =/ "STATUS" SP mailbox SP "(" [status-att SP status-value *(SP status-att SP status-value)] ")" status-value = number /modtime-valuemod-sequence-value store = "STORE" SP set store-modifiers SP store-att-flags store-modifiers = [ SP "(" 1*store-modifier ")" ] store-modifier = "UNCHANGEDSINCE" SPmodtime-valuemod-sequence-value fetch-att =/fetch-modtimefetch-mod-sequence ;; modifies original IMAP4 fetch-attfetch-modtimefetch-mod-sequence ="MODTIME""MODSEQ" SP entry-names fetch-mod-resp ="MODTIME""MODSEQ" SP "("permessage-modtimepermsg-modsequence *(entry-name SPmodtime-value)mod-sequence-value) ")" search-key =/search-modtimesearch-modsequence ;; modifies original IMAP4 search-keysearch-modtimesearch-modsequence ="MODTIME""MODSEQ" SP entry-name SPmodtime-valuemod-sequence-value resp-text-code =/ "HIGHESTMODSEQ" SP mod-sequence-value / "MODIFIED" SP set / "MODSEQ" SP set SP mod-sequence-value ;; set of message numbers forSTORESTORE/FETCH or ;; set of UIDs for UIDSTORESTORE/UID FECTH entry-names = "(" *entry-name ")" ;; empty list means that only per-messageMODTIMEMODSEQ should be returned entry-name = '"' "/message/flags/system/" attr-flag '"' ;; each system or user defined flag <flag> is mapped to ;; "/message/flags/system/<flag>". ;; system IMAP flags must have two leading "\", ;; because "\" is an escape character.permessage-modtimepermsg-modsequence =modtime-valuemod-sequence-value ;; per messagemodtime,mod-sequence, if server supports per flagmodtimes,mod-sequences, ;; this is the highestmodtimemod-sequence between all metadata itemsmodtime-valuemod-sequence-value =time mailbox-data =/ "MODTIME" SP set SP modtime-value1*DIGIT ;; Unsigned 64-bit integer (mod-sequence) ;; (0 <= n < 18,446,744,073,709,551,615) <<Borrowed from IMAP4rev1 and modified accordingly:>> attr-flag = "\\Answered" / "\\Flagged" / "\\Deleted" / "\\Seen" / "\\Draft" / attr-flag-keyword / attr-flag-extension ;; Does not include "\Recent" attr-flag-extension = "\\" atom ;; Future expansion. Client implementations ;; MUST accept flag-extension flags. Server ;; implementations MUST NOT generate ;; flag-extension flags except as defined by ;; future standard or standards-track ;; revisions of this specification. attr-flag-keyword = atom<<Borrowed from ACAP:>> time = <"> time-year time-month time-day time-hour time-minute time-second time-subsecond <"> ;; Timestamp in UTC time-day = 2DIGIT ;; 01-31 time-hour = 2DIGIT ;; 00-23 time-minute = 2DIGIT ;; 00-59 time-month = 2DIGIT ;; 01-12 time-second = 2DIGIT ;; 00-60 time-subsecond = *DIGIT time-year = 4DIGIT6. Security Considerations There are no known security issues with this extension. 7. References [ABNF] Crocker, Overell, "Augmented BNF for Syntax Specifications: ABNF", RFC 2234, Internet Mail Consortium, Demon Internet Ltd, November 1997. [IMAP4] Crispin, M., "Internet Message Access Protocol - Version 4rev1", RFC 2060, University of Washington, December 1996. [KEYWORDS] Bradner, "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, Harvard University, March 1997. [ACAP] Newman, Myers, "ACAP -- Application Configuration Access Protocol", RFC 2244, Innosoft, Netscape, November 1997. <ftp://ftp.isi.edu/in-notes/rfc2244.txt> [ANNOTATION] Gellens, R., Daboo, C., "IMAP ANNOTATE Extension", work in progress. <http://www.ietf.org/internet-drafts/draft-ietf-imapext-annotate-xx.txt> [SORT-EXT] Crispin, M., "Internet Message Access Protocol -- SORT Extension", work in progress. <http://www.ietf.org/internet-drafts/draft-crispin-imapext-sort-xx.txt> 8. AcknowledgmentsMany thanks to Randall Gellens for his comments on how MODTIME should work together with ANNOTATE extension.Some text was borrowed from "IMAP ANNOTATE Extension" by Randall Gellens and Cyrus Daboo and "ACAP -- Application Configuration Access Protocol" by Chris Newman and John Myers. Many thanks to Randall Gellens for his comments on how CONDSTORE should interact with ANNOTATE extension. Authors also acknowledge the feedback provided by Cyrus Daboo and Larry Greenfield. 8. Full Copyright Statement Copyright (C) The Internet Society 2001. All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. ----