httpc - An HTTP/1.1 client
Please see following description for synopsis
httpc(3) Erlang Module Definition httpc(3)
NAME
httpc - An HTTP/1.1 client
DESCRIPTION
This module provides the API to an HTTP/1.1 compatible client according
to RFC 2616. Caching is not supported.
Note:
When starting the Inets application, a manager process for the default
profile is started. The functions in this API that do not explicitly
use a profile accesses the default profile. A profile keeps track of
proxy options, cookies, and other options that can be applied to more
than one request.
If the scheme https is used, the SSL application must be started. When
https links need to go through a proxy, the CONNECT method extension to
HTTP-1.1 is used to establish a tunnel and then the connection is
upgraded to TLS. However, "TLS upgrade" according to RFC 2817is not
supported.
Pipelining is only used if the pipeline time-out is set, otherwise per-
sistent connections without pipelining are used. That is, the client
always waits for the previous response before sending the next request.
Some examples are provided in the Inets User's Guide.
DATA TYPES
Type definitions that are used more than once in this module:
boolean() = true | false
http_string() = list of ASCII characters
request_id() = reference()
profile() = atom()
path() = string() representing a file path or directory path
ip_address() = See the inet(3) manual page in Kernel.
socket_opt() = See the options used by gen_tcp(3) gen_tcp(3) and ssl(3)
connect(s)
HTTP DATA TYPES
Type definitions related to HTTP:
method() = head | get | put | post | trace | options | delete | patch
request():
= {url(), headers()}
| {url(), headers(), content_type(), body()}
url() = http_string() syntax according to the URI definition in RFC
3986, for example "http://www.erlang.org"
Warning:
Please note that httpc normalizes input URIs before internal processing
and special care shall be taken when the URI has percent ("%") charac-
ters. A percent serves as the indicator for percent-encoded octets and
it must be percent-encoded as "%25" for that octet to be used as data
within the URI.
For example, in order to send an HTTP GET request with the URI
http://localhost/foo%25bar, the percent character must be percent-
encoded when creating the request: httpc:request("http://local-
host/foo%2525bar").
status_line() = {http_version(), status_code(), reason_phrase()}
http_version() = http_string(), for example, "HTTP/1.1"
status_code() = integer()
reason_phrase() = string()
content_type() = http_string()
headers() = [header()]
header() = {field(), value()}
field() = [byte()]
value() = binary() | iolist()
body():
= http_string() | binary()
| {fun(accumulator())
-> body_processing_result(), accumulator()}
| {chunkify, fun(accumulator())
-> body_processing_result(), accumulator()}
body_processing_result() = eof | {ok, iolist(), accumulator()}
accumulator() = term()
filename() = string()
For more information about HTTP, see RFC 2616.
SSL DATA TYPES
See ssl(3) for information about SSL options (ssloptions()).
HTTP CLIENT SERVICE START/STOP
An HTTP client can be configured to start when starting the Inets
application or started dynamically in runtime by calling the Inets
application API inets:start(httpc, ServiceConfig) or inets:start(httpc,
ServiceConfig, How), see inets(3). The configuration options are as
follows:
{profile, profile()}:
Name of the profile, see DATA TYPES. This option is mandatory.
{data_dir, path()}:
Directory where the profile can save persistent data. If omitted,
all cookies are treated as session cookies.
The client can be stopped using inets:stop(httpc, Pid) or
inets:stop(httpc, Profile).
EXPORTS
cancel_request(RequestId) ->
cancel_request(RequestId, Profile) -> ok
Types:
RequestId = request_id() - A unique identifier as returned by
request/4
Profile = profile() | pid()
When started stand_alone only the pid can be used.
Cancels an asynchronous HTTP request. Notice that this does not
guarantee that the request response is not delivered. Because it
is asynchronous, the request can already have been completed
when the cancellation arrives.
cookie_header(Url) ->
cookie_header(Url, Profile | Opts) -> header() | {error, Reason}
cookie_header(Url, Opts, Profile) -> header() | {error, Reason}
Types:
Url = url()
Opts = [cookie_header_opt()]
Profile = profile() | pid()
When started stand_alone.
cookie_header_opt() = {ipv6_host_with_brackets, boolean()}
Returns the cookie header that would have been sent when making
a request to Url using profile Profile. If no profile is speci-
fied, the default profile is used.
Option ipv6_host_with_bracket deals with how to parse IPv6
addresses. For details, see argument Options of request/[4,5].
get_options(OptionItems) -> {ok, Values} | {error, Reason}
get_options(OptionItems, Profile) -> {ok, Values} | {error, Reason}
Types:
OptionItems = all | [option_item()]
option_item() = proxy | https_proxy | max_sessions |
keep_alive_timeout | max_keep_alive_length | pipeline_timeout
| max_pipeline_length | cookies | ipfamily | ip | port |
socket_opts | verbose | unix_socket
Profile = profile() | pid()
When started stand_alone only the pid can used.
Values = [{option_item(), term()}]
Reason = term()
Retrieves the options currently used by the client.
info() -> list()
info(Profile) -> list()
Types:
Profile = profile() | pid()
When started stand_alone only the pid can be used.
Produces a list of miscellaneous information. Intended for
debugging. If no profile is specified, the default profile is
used.
reset_cookies() -> void()
reset_cookies(Profile) -> void()
Types:
Profile = profile() | pid()
When started stand_alone only the pid can be used.
Resets (clears) the cookie database for the specified Profile.
If no profile is specified the default profile is used.
request(Url) ->
request(Url, Profile) -> {ok, Result} | {error, Reason}
Types:
Url = url()
Result = {status_line(), headers(), Body} | {status_code(),
Body} | request_id()
Body = http_string() | binary()
Profile = profile() | pid()
When started stand_alone only the pid can be used.
Reason = term()
Equivalent to httpc:request(get, {Url, []}, [], []).
request(Method, Request, HTTPOptions, Options) ->
request(Method, Request, HTTPOptions, Options, Profile) -> {ok, Result}
| {ok, saved_to_file} | {error, Reason}
Types:
Method = method()
Request = request()
HTTPOptions = http_options()
http_options() = [http_option()]
http_option() = {timeout, timeout()} | {connect_timeout,
timeout()} | {ssl, ssloptions()} | {essl, ssloptions()} |
{autoredirect, boolean()} | {proxy_auth, {userstring(), pass-
wordstring()}} | {version, http_version()} | {relaxed, bool-
ean()}
timeout() = integer() >= 0 | infinity
Options = options()
options() = [option()]
option() = {sync, boolean()} | {stream, stream_to()} |
{body_format, body_format()} | {full_result, boolean()} |
{headers_as_is, boolean() | {socket_opts, socket_opts()} |
{receiver, receiver()} | {ipv6_host_with_brackets, boolean()}
stream_to() = none | self | {self, once} | filename()
socket_opts() = [socket_opt()]
receiver() = pid() | function()/1 | {Module, Function, Args}
Module = atom()
Function = atom()
Args = list()
body_format() = string | binary
Result = {status_line(), headers(), Body} | {status_code(),
Body} | request_id()
Body = http_string() | binary()
Profile = profile() | pid()
When started stand_alone only the pid can be used.
Reason = term()
Sends an HTTP request. The function can be both synchronous and
asynchronous. In the latter case, the function returns {ok,
RequestId} and then the information is delivered to the receiver
depending on that value.
HTTP option (http_option()) details:
timeout:
Time-out time for the request.
The clock starts ticking when the request is sent.
Time is in milliseconds.
Default is infinity.
connect_timeout:
Connection time-out time, used during the initial request,
when the client is connecting to the server.
Time is in milliseconds.
Default is the value of option timeout.
ssl:
This is the SSL/TLS connecting configuration option.
Defaults to []. See ssl:connect/[2,3,4] for available
options.
autoredirect:
The client automatically retrieves the information from the
new URI and returns that as the result, instead of a 30X-
result code.
For some 30X-result codes, automatic redirect is not
allowed. In these cases the 30X-result is always returned.
Default is true.
proxy_auth:
A proxy-authorization header using the provided username and
password is added to the request.
version:
Can be used to make the client act as an HTTP/1.0 client. By
default this is an HTTP/1.1 client. When using HTTP/1.0 per-
sistent connections are not used.
Default is the string "HTTP/1.1".
relaxed:
If set to true, workarounds for known server deviations from
the HTTP-standard are enabled.
Default is false.
Option (option()) details:
sync:
Option for the request to be synchronous or asynchronous.
Default is true.
stream:
Streams the body of a 200 or 206 response to the calling
process or to a file. When streaming to the calling process
using option self, the following stream messages are sent to
that process: {http, {RequestId, stream_start, Headers}},
{http, {RequestId, stream, BinBodyPart}}, and {http,
{RequestId, stream_end, Headers}}.
When streaming to the calling processes using option {self,
once}, the first message has an extra element, that is,
{http, {RequestId, stream_start, Headers, Pid}}. This is the
process id to be used as an argument to httpc:stream_next/1
to trigger the next message to be sent to the calling
process.
Notice that chunked encoding can add headers so that there
are more headers in the stream_end message than in
stream_start. When streaming to a file and the request is
asynchronous, the message {http, {RequestId, saved_to_file}}
is sent.
Default is none.
body_format:
Defines if the body is to be delivered as a string or
binary. This option is only valid for the synchronous
request.
Default is string.
full_result:
Defines if a "full result" is to be returned to the caller
(that is, the body, the headers, and the entire status line)
or not (the body and the status code).
Default is true.
headers_as_is:
Defines if the headers provided by the user are to be made
lower case or to be regarded as case sensitive.
The HTTP standard requires them to be case insensitive. Use
this feature only if there is no other way to communicate
with the server or for testing purpose. When this option is
used, no headers are automatically added. All necessary
headers must be provided by the user.
Default is false.
socket_opts:
Socket options to be used for this request.
Overrides any value set by function set_options.
The validity of the options is not checked by the HTTP
client they are assumed to be correct and passed on to ssl
application and inet driver, which may reject them if they
are not correct.
Note:
Persistent connections are not supported when setting the
socket_opts option. When socket_opts is not set the current
implementation assumes the requests to the same host, port
combination will use the same socket options.
By default the socket options set by function
set_options/[1,2] are used when establishing a connection.
receiver:
Defines how the client delivers the result of an asynchro-
nous request (sync has the value false).
pid():
Messages are sent to this process in the format {http,
ReplyInfo}.
function/1:
Information is delivered to the receiver through calls to
the provided fun Receiver(ReplyInfo).
{Module, Function, Args}:
Information is delivered to the receiver through calls to
the callback function apply(Module, Function, [ReplyInfo |
Args]).
In all of these cases, ReplyInfo has the following struc-
ture:
{RequestId, saved_to_file}
{RequestId, {error, Reason}}
{RequestId, Result}
{RequestId, stream_start, Headers}
{RequestId, stream_start, Headers, HandlerPid}
{RequestId, stream, BinBodyPart}
{RequestId, stream_end, Headers}
Default is the pid of the process calling the request func-
tion (self()).
ipv6_host_with_brackets:
Defines when parsing the Host-Port part of an URI with an
IPv6 address with brackets, if those brackets are to be
retained (true) or stripped (false).
Default is false.
set_options(Options) ->
set_options(Options, Profile) -> ok | {error, Reason}
Types:
Options = [Option]
Option = {proxy, {Proxy, NoProxy}}
| {https_proxy, {Proxy, NoProxy}}
| {max_sessions, MaxSessions}
| {max_keep_alive_length, MaxKeepAlive}
| {keep_alive_timeout, KeepAliveTimeout}
| {max_pipeline_length, MaxPipeline}
| {pipeline_timeout, PipelineTimeout}
| {cookies, CookieMode}
| {ipfamily, IpFamily}
| {ip, IpAddress}
| {port, Port}
| {socket_opts, socket_opts()}
| {verbose, VerboseMode}
| {unix_socket, UnixSocket}
Proxy = {Hostname, Port}
Hostname = http_string()
Example: "localhost" or "foo.bar.se"
Port = integer()
Example: 8080
NoProxy = [NoProxyDesc]
NoProxyDesc = DomainDesc | HostName | IPDesc
DomainDesc = "*.Domain"
Example: "*.ericsson.se"
IpDesc = http_string()
Example: "134.138" or "[FEDC:BA98" (all IP addresses start-
ing with 134.138 or FEDC:BA98), "66.35.250.150" or
"[2010:836B:4179::836B:4179]" (a complete IP address).
proxy defaults to {undefined, []}, that is, no proxy is
configured and https_proxy defaults to the value of proxy.
MaxSessions = integer()
Maximum number of persistent connections to a host. Default
is 2.
MaxKeepAlive = integer()
Maximum number of outstanding requests on the same connec-
tion to a host. Default is 5.
KeepAliveTimeout = integer()
If a persistent connection is idle longer than the
keep_alive_timeout in milliseconds, the client closes the
connection. The server can also have such a time-out but do
not take that for granted. Default is 120000 (= 2 min).
MaxPipeline = integer()
Maximum number of outstanding requests on a pipelined con-
nection to a host. Default is 2.
PipelineTimeout = integer()
If a persistent connection is idle longer than the pipe-
line_timeout in milliseconds, the client closes the connec-
tion. Default is 0, which results in pipelining not being
used.
CookieMode = enabled | disabled | verify
If cookies are enabled, all valid cookies are automatically
saved in the cookie database of the client manager. If
option verify is used, function store_cookies/2 has to be
called for the cookies to be saved. Default is disabled.
IpFamily = inet | inet6 | local
Default is inet.
IpAddress = ip_address()
If the host has several network interfaces, this option
specifies which one to use. See gen_tcp:connect/3,4 for
details.
Port = integer()
Local port number to use. See gen_tcp:connect/3,4 for
details.
socket_opts() = [socket_opt()]
The options are appended to the socket options used by the
client. These are the default values when a new request
handler is started (for the initial connect). They are
passed directly to the underlying transport (gen_tcp or
SSL) without verification.
VerboseMode = false | verbose | debug | trace
Default is false. This option is used to switch on (or off)
different levels of Erlang trace on the client. It is a
debug feature.
Profile = profile() | pid()
When started stand_alone only the pid can be used.
UnixSocket = path()
Experimental option for sending HTTP requests over a unix
domain socket. The value of unix_socket shall be the full
path to a unix domain socket file with read/write permis-
sions for the erlang process. Default is undefined.
Sets options to be used for subsequent requests.
Note:
If possible, the client keeps its connections alive and uses
persistent connections with or without pipeline depending on
configuration and current circumstances. The HTTP/1.1 specifica-
tion does not provide a guideline for how many requests that are
ideal to be sent on a persistent connection. This depends much
on the application.
A long queue of requests can cause a user-perceived delay, as
earlier requests can take a long time to complete. The HTTP/1.1
specification suggests a limit of two persistent connections per
server, which is the default value of option max_sessions.
The current implementation assumes the requests to the same
host, port combination will use the same socket options.
store_cookies(SetCookieHeaders, Url) ->
store_cookies(SetCookieHeaders, Url, Profile) -> ok | {error, Reason}
Types:
SetCookieHeaders = headers() - where field = "set-cookie"
Url = url()
Profile = profile() | pid()
When started stand_alone only the pid can be used.
Saves the cookies defined in SetCookieHeaders in the client pro-
file cookie database. Call this function if option cookies is
set to verify. If no profile is specified, the default profile
is used.
stream_next(Pid) -> ok
Types:
Pid = pid()
As received in the stream_start message
Triggers the next message to be streamed, that is, the same
behavior as active ones for sockets.
which_cookies() -> cookies()
which_cookies(Profile) -> cookies()
Types:
Profile = profile() | pid()
When started stand_alone only the pid can be used.
cookies() = [cookie_stores()]
cookie_stores() = {cookies, cookies()} | {session_cookies,
cookies()}
cookies() = [cookie()]
cookie() = term()
Produces a list of the entire cookie database. Intended for
debugging/testing purposes. If no profile is specified, the
default profile is used.
which_sessions() -> session_info()
which_sessions(Profile) -> session_info()
Types:
Profile = profile() | pid()
When started stand_alone only the pid can be used.
session_info() = {[session()], [term()], [term()]}
session() = term() - Internal representation of a session
This function is intended for debugging only. It produces a
slightly processed dump of the session database. The first list
of the session information tuple will contain session informa-
tion on an internal format. The last two lists of the session
information tuple should always be empty if the code is working
as intended. If no profile is specified, the default profile is
used.
SEE ALSO
RFC 2616, inets(3), gen_tcp(3), ssl(3)
Ericsson AB inets 7.5 httpc(3)