From 8e20c03d3a1d2f95985fad85f8ea923e75d9f87a Mon Sep 17 00:00:00 2001 From: "Alan M. Carroll" Date: Fri, 7 Aug 2015 12:32:35 -0500 Subject: [PATCH 1/3] API: Proposal for additional IO buffer and body template API. --- .../api/TSHttpTxnApplyLogFormat.en.rst | 40 +++++++ .../api/TSHttpTxnErrorBodyBufferSet.en.rst | 37 +++++++ .../api/TSIOBufferReaderAlloc.en.rst | 104 ++++++++++++++++++ 3 files changed, 181 insertions(+) create mode 100644 doc/reference/api/TSHttpTxnApplyLogFormat.en.rst create mode 100644 doc/reference/api/TSHttpTxnErrorBodyBufferSet.en.rst create mode 100644 doc/reference/api/TSIOBufferReaderAlloc.en.rst diff --git a/doc/reference/api/TSHttpTxnApplyLogFormat.en.rst b/doc/reference/api/TSHttpTxnApplyLogFormat.en.rst new file mode 100644 index 00000000000..b79e8a0a4ed --- /dev/null +++ b/doc/reference/api/TSHttpTxnApplyLogFormat.en.rst @@ -0,0 +1,40 @@ +.. Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed + with this work for additional information regarding copyright + ownership. The ASF licenses this file to you under the Apache + License, Version 2.0 (the "License"); you may not use this file + except in compliance with the License. You may obtain a copy of + the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied. See the License for the specific language governing + permissions and limitations under the License. + + +TSHttpTxnApplyLogFormat +======================= + +Synopsis +-------- + +`#include ` + +.. c:function:: TSReturnCode TSHttpTxnApplyLogFormat(TSHttpTxn txnp, char const* format, TSIOBuffer out) + + +Description +----------- + +Apply the log :arg:`format` string to the transaction :arg:`txnp` placing the result in the buffer +:arg:`out`. The format is identical to :ref:`custom log formats `. The result +is appended to :arg:`out` in ASCII format. :arg:`out` must be created by the plugin. + +See also +======== + +:manpage:`TSIOBufferCreate(3ts)`, :manpage:`TSIOBufferReaderAlloc(3ts)` + diff --git a/doc/reference/api/TSHttpTxnErrorBodyBufferSet.en.rst b/doc/reference/api/TSHttpTxnErrorBodyBufferSet.en.rst new file mode 100644 index 00000000000..e02c448629f --- /dev/null +++ b/doc/reference/api/TSHttpTxnErrorBodyBufferSet.en.rst @@ -0,0 +1,37 @@ +.. Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed + with this work for additional information regarding copyright + ownership. The ASF licenses this file to you under the Apache + License, Version 2.0 (the "License"); you may not use this file + except in compliance with the License. You may obtain a copy of + the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied. See the License for the specific language governing + permissions and limitations under the License. + + +TSHttpTxnErrorBodyBufferSet +=========================== + +Sets an error type body to a transaction. + + +Synopsis +-------- + +`#include ` + +.. c:function:: void TSHttpTxnErrorBodyBufferSet(TSHttpTxn txnp, TSIOBufferReader body, char const* mimetype) + + +Description +----------- + +Note that both string arguments must be allocated with :c:func:`TSmalloc` or :c:func:`TSstrdup`. +The mimetype argument is optional, if not provided it defaults to "text/html". Sending an emptry +string would prevent setting a content type header (but that is not advised). diff --git a/doc/reference/api/TSIOBufferReaderAlloc.en.rst b/doc/reference/api/TSIOBufferReaderAlloc.en.rst new file mode 100644 index 00000000000..5781574bf70 --- /dev/null +++ b/doc/reference/api/TSIOBufferReaderAlloc.en.rst @@ -0,0 +1,104 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +.. default-domain:: c + +================== +TSIOBufferReader +================== + +Traffic Server IO buffer reader API. + +Synopsis +======== +`#include ` + +.. function:: TSIOBufferReader TSIOBufferReaderAlloc(TSIOBuffer bufp) +.. function:: TSIOBufferReader TSIOBufferReaderClone(TSIOBufferReader readerp); +.. function:: void TSIOBufferReaderFree(TSIOBufferReader readerp); +.. function:: void TSIOBufferReaderConsume(TSIOBufferReader readerp, int64_t nbytes); +.. function:: TSIOBufferBlock TSIOBufferReaderStart(TSIOBufferReader readerp); +.. function:: int64_t TSIOBufferReaderAvail(TSIOBufferReader readerp); +.. function:: int64_t TSIOBufferReaderRead(TSIOBufferReader reader, const void * buf, int64_t length) +.. function:: bool TSIOBufferReaderIterate(TSIOBufferReader reader, TSIOBufferBlockFunc* func, void* context) + +.. :type:: TSIOBufferBlockFunc + + ``bool (*TSIOBufferBlockFunc)(void const* data, size_t nbytes, void* context)`` + + :arg:`data` is the data in the :type:`TSIOBufferBlock` of size :arg:`nbytes`. :arg:`context` is + opaque data provided to the API call along with this function and passed on to the function. This + function should return ``true`` to continue iterationa and ``false`` to terminate iteration. + +Description +=========== + +The :type:`TSIOBufferReader` is an read accessor for :type:`TSIOBuffer`. It represents a location in +the contents of the buffer. A buffer can have multiple readers and each reader consumes data in the +buffer independently. Data which for which there are no readers is discarded from the buffer. This +has two very important consequences -- + +* Data for which there are no readers and no writer will be discarded. In effect this means without + any readers only the current write buffer block will be maintained, older buffer blocks will + disappear. +* Conversely keeping a reader around unused will pin the buffer data in memory. This can be useful or harmful. + +A buffer has a fixed amount of possible readers (currently 5) which is determined at compile +time. Reader allocation is fast and cheap until this maxium is reached at which point it fails. + +:func:`TSIOBufferReaderAlloc` allocates a reader for the IO buffer :arg:`bufp`. This should only be + called on a newly allocated buffer. If not the location of the reader in the buffer will be + indeterminate. Use :func:`TSIOBufferReaderClone` to get another reader if the buffer is + already in use. + +:func:`TSIOBufferReaderClone` allocates a reader and sets its position in the buffer to be the same as :arg:`reader`. + +:func:`TSIOBufferReaderFree` de-allocates the reader. Any data referenced only by this reader is + discared from the buffer. + +:func:`TSIOBufferReaderConsume` advances the position :arg:`reader` in its IO buffer by the + the smaller of :arg:`bytes` and the maximum available in its IO buffer. + +:func:`TSIOBufferReaderStart` returns the IO buffer block containing the position of +:arg:`reader`. Note the position of :arg:`reader` is in the block but not necessarily the first byte +of the block. + +:func:`TSIOBufferReaderAvail` returns the number of bytes in the buffer which :arg:`reader` could + consume. That is the number of bytes in the IO buffer past the current position of + :arg:`reader`. + +:func:`TSIOBufferReaderRead` reads data from :arg:`reader`. This first copies data from the IO + buffer for :arg:`reader` to the target buffer :arg:`bufp`, starting at :arg:`reader`s + position, and then advances (as with :func:`TSIOBufferReaderConsume`) :arg:`reader`s + position past the copied data. The amount of data read in this fashion is the smaller of the + amount of data available in the IO buffer for :arg:`reader` and the size of the target buffer + (:arg:`length`). + +:func:`TSIOBufferReaderIterate` iterates over the blocks for :arg:`reader`. For each block +:arg:`func` is called with with the data for the block and :arg:`context`. The :arg:`context` is an +opaque type to this function and is passed unchanged to :arg:`func`. It is intended to be used as +context for :arg:`func`. If :arg:`func` returns ``false`` the iteration terminates. The return value +is the return value from the last call to :arg:`func`. + +.. note:: Destroying a :type:`TSIOBuffer` will de-allocate and destroy all readers for that buffer. + + + +See also +======== + +:manpage:`TSIOBufferCreate(3ts)` From 433bb2a012ee2911e140c043c23c55d1bfd5624f Mon Sep 17 00:00:00 2001 From: "Alan M. Carroll" Date: Fri, 7 Aug 2015 14:03:59 -0500 Subject: [PATCH 2/3] Tighten up some definitions and explanations. --- .../api/TSIOBufferReaderAlloc.en.rst | 51 ++++++++++++------- 1 file changed, 32 insertions(+), 19 deletions(-) diff --git a/doc/reference/api/TSIOBufferReaderAlloc.en.rst b/doc/reference/api/TSIOBufferReaderAlloc.en.rst index 5781574bf70..ad97bbc681f 100644 --- a/doc/reference/api/TSIOBufferReaderAlloc.en.rst +++ b/doc/reference/api/TSIOBufferReaderAlloc.en.rst @@ -28,26 +28,27 @@ Synopsis `#include ` .. function:: TSIOBufferReader TSIOBufferReaderAlloc(TSIOBuffer bufp) -.. function:: TSIOBufferReader TSIOBufferReaderClone(TSIOBufferReader readerp); -.. function:: void TSIOBufferReaderFree(TSIOBufferReader readerp); -.. function:: void TSIOBufferReaderConsume(TSIOBufferReader readerp, int64_t nbytes); -.. function:: TSIOBufferBlock TSIOBufferReaderStart(TSIOBufferReader readerp); -.. function:: int64_t TSIOBufferReaderAvail(TSIOBufferReader readerp); +.. function:: TSIOBufferReader TSIOBufferReaderClone(TSIOBufferReader readerp) +.. function:: void TSIOBufferReaderFree(TSIOBufferReader readerp) +.. function:: void TSIOBufferReaderConsume(TSIOBufferReader readerp, int64_t nbytes) +.. function:: TSIOBufferBlock TSIOBufferReaderStart(TSIOBufferReader readerp) +.. function:: int64_t TSIOBufferReaderAvail(TSIOBufferReader readerp) +.. function:: bool TSIOBufferReaderIsAvailAtLeast(TSIOBufferReader, int64_t nbytes) .. function:: int64_t TSIOBufferReaderRead(TSIOBufferReader reader, const void * buf, int64_t length) .. function:: bool TSIOBufferReaderIterate(TSIOBufferReader reader, TSIOBufferBlockFunc* func, void* context) .. :type:: TSIOBufferBlockFunc - ``bool (*TSIOBufferBlockFunc)(void const* data, size_t nbytes, void* context)`` + ``bool (*TSIOBufferBlockFunc)(void const* data, int64_t nbytes, void* context)`` - :arg:`data` is the data in the :type:`TSIOBufferBlock` of size :arg:`nbytes`. :arg:`context` is + :arg:`data` is the data in the :type:`TSIOBufferBlock` and is :arg:`nbytes` long. :arg:`context` is opaque data provided to the API call along with this function and passed on to the function. This - function should return ``true`` to continue iterationa and ``false`` to terminate iteration. + function should return ``true`` to continue iteration or ``false`` to terminate iteration. Description =========== -The :type:`TSIOBufferReader` is an read accessor for :type:`TSIOBuffer`. It represents a location in +:type:`TSIOBufferReader` is an read accessor for :type:`TSIOBuffer`. It represents a location in the contents of the buffer. A buffer can have multiple readers and each reader consumes data in the buffer independently. Data which for which there are no readers is discarded from the buffer. This has two very important consequences -- @@ -68,18 +69,24 @@ time. Reader allocation is fast and cheap until this maxium is reached at which :func:`TSIOBufferReaderClone` allocates a reader and sets its position in the buffer to be the same as :arg:`reader`. :func:`TSIOBufferReaderFree` de-allocates the reader. Any data referenced only by this reader is - discared from the buffer. + discarded from the buffer. -:func:`TSIOBufferReaderConsume` advances the position :arg:`reader` in its IO buffer by the - the smaller of :arg:`bytes` and the maximum available in its IO buffer. +:func:`TSIOBufferReaderConsume` advances the position of :arg:`reader` in its IO buffer by the + the smaller of :arg:`nbytes` and the maximum available in the buffer. :func:`TSIOBufferReaderStart` returns the IO buffer block containing the position of -:arg:`reader`. Note the position of :arg:`reader` is in the block but not necessarily the first byte -of the block. +:arg:`reader`. -:func:`TSIOBufferReaderAvail` returns the number of bytes in the buffer which :arg:`reader` could - consume. That is the number of bytes in the IO buffer past the current position of - :arg:`reader`. + .. note:: The byte at the position of :arg:`reader` is in the block but is not necessarily the first byte of the block. + +:func:`TSIOBufferReaderAvail` returns the number of bytes which :arg:`reader` could consume. That is + the number of bytes in the IO buffer starting at the current position of :arg:`reader`. + +:func:`TSIOBufferReaderIsAvailAtLeast` return ``true`` if the available number of bytes for + :arg:`reader` is at least :arg:`nbytes`, ``false`` if not. This can be more efficient than + :func:`TSIOBufferReadAvail` because the latter must walk all the IO buffer blocks in the IO + buffer. This function returns as soon as the return value can be determined. In particular a + value of ``1`` for :arg:`nbytes` means only the first buffer block will be checked. :func:`TSIOBufferReaderRead` reads data from :arg:`reader`. This first copies data from the IO buffer for :arg:`reader` to the target buffer :arg:`bufp`, starting at :arg:`reader`s @@ -91,8 +98,14 @@ of the block. :func:`TSIOBufferReaderIterate` iterates over the blocks for :arg:`reader`. For each block :arg:`func` is called with with the data for the block and :arg:`context`. The :arg:`context` is an opaque type to this function and is passed unchanged to :arg:`func`. It is intended to be used as -context for :arg:`func`. If :arg:`func` returns ``false`` the iteration terminates. The return value -is the return value from the last call to :arg:`func`. +context for :arg:`func`. If :arg:`func` returns ``false`` the iteration terminates. If :arg:`func` +returns true the block is consumed. The return value for :func:`TSIOBufferReaderIterate` is the +return value from the last call to :arg:`func`. + + .. note:: If it would be a problem for the iteration to consume the data (especially in cases where + ``false`` might be returned) the reader can be cloned via :func:`TSIOBufferReaderClone` to + keep the data in the IO buffer and available. If not needed the reader can be destroyed or + if needed the original reader can be destroyed and replaced by the clone. .. note:: Destroying a :type:`TSIOBuffer` will de-allocate and destroy all readers for that buffer. From be0772f9b636c3464e06948ce79a27fdf3a69f5d Mon Sep 17 00:00:00 2001 From: "Alan M. Carroll" Date: Fri, 7 Aug 2015 14:19:14 -0500 Subject: [PATCH 3/3] Updated description of TSHttpTxnErrorBodyBufferSet --- doc/reference/api/TSHttpTxnErrorBodyBufferSet.en.rst | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/doc/reference/api/TSHttpTxnErrorBodyBufferSet.en.rst b/doc/reference/api/TSHttpTxnErrorBodyBufferSet.en.rst index e02c448629f..9f762ecd5df 100644 --- a/doc/reference/api/TSHttpTxnErrorBodyBufferSet.en.rst +++ b/doc/reference/api/TSHttpTxnErrorBodyBufferSet.en.rst @@ -14,6 +14,7 @@ implied. See the License for the specific language governing permissions and limitations under the License. +.. default-domain:: c TSHttpTxnErrorBodyBufferSet =========================== @@ -26,12 +27,14 @@ Synopsis `#include ` -.. c:function:: void TSHttpTxnErrorBodyBufferSet(TSHttpTxn txnp, TSIOBufferReader body, char const* mimetype) +.. function:: void TSHttpTxnErrorBodyBufferSet(TSHttpTxn txnp, TSIOBufferReader body, char const* mimetype) Description ----------- -Note that both string arguments must be allocated with :c:func:`TSmalloc` or :c:func:`TSstrdup`. +This sets the body for the response to the user agent when the status is an error code. + +:arg:`mimetype` must be allocated with :func:`TSmalloc` or :func:`TSstrdup`. The mimetype argument is optional, if not provided it defaults to "text/html". Sending an emptry string would prevent setting a content type header (but that is not advised).