request @ 2.45.0 - README.md
1
# Request — Simplified HTTP client
2
3
[![NPM](https://nodei.co/npm/request.png?downloads=true&downloadRank=true&stars=true)](https://nodei.co/npm/request/)
4
5
## Super simple to use
6
7
Request is designed to be the simplest way possible to make http calls. It supports HTTPS and follows redirects by default.
8
9
```javascript
10
var request = require('request');
11
request('http://www.google.com', function (error, response, body) {
12
if (!error && response.statusCode == 200) {
13
console.log(body) // Print the google web page.
14
}
15
})
16
```
17
18
## Streaming
19
20
You can stream any response to a file stream.
21
22
```javascript
23
request('http://google.com/doodle.png').pipe(fs.createWriteStream('doodle.png'))
24
```
25
26
You can also stream a file to a PUT or POST request. This method will also check the file extension against a mapping of file extensions to content-types (in this case `application/json`) and use the proper `content-type` in the PUT request (if the headers don’t already provide one).
27
28
```javascript
29
fs.createReadStream('file.json').pipe(request.put('http://mysite.com/obj.json'))
30
```
31
32
Request can also `pipe` to itself. When doing so, `content-type` and `content-length` are preserved in the PUT headers.
33
34
```javascript
35
request.get('http://google.com/img.png').pipe(request.put('http://mysite.com/img.png'))
36
```
37
38
Now let’s get fancy.
39
40
```javascript
41
http.createServer(function (req, resp) {
42
if (req.url === '/doodle.png') {
43
if (req.method === 'PUT') {
44
req.pipe(request.put('http://mysite.com/doodle.png'))
45
} else if (req.method === 'GET' || req.method === 'HEAD') {
46
request.get('http://mysite.com/doodle.png').pipe(resp)
47
}
48
}
49
})
50
```
51
52
You can also `pipe()` from `http.ServerRequest` instances, as well as to `http.ServerResponse` instances. The HTTP method, headers, and entity-body data will be sent. Which means that, if you don't really care about security, you can do:
53
54
```javascript
55
http.createServer(function (req, resp) {
56
if (req.url === '/doodle.png') {
57
var x = request('http://mysite.com/doodle.png')
58
req.pipe(x)
59
x.pipe(resp)
60
}
61
})
62
```
63
64
And since `pipe()` returns the destination stream in ≥ Node 0.5.x you can do one line proxying. :)
65
66
```javascript
67
req.pipe(request('http://mysite.com/doodle.png')).pipe(resp)
68
```
69
70
Also, none of this new functionality conflicts with requests previous features, it just expands them.
71
72
```javascript
73
var r = request.defaults({'proxy':'http://localproxy.com'})
74
75
http.createServer(function (req, resp) {
76
if (req.url === '/doodle.png') {
77
r.get('http://google.com/doodle.png').pipe(resp)
78
}
79
})
80
```
81
82
You can still use intermediate proxies, the requests will still follow HTTP forwards, etc.
83
84
## Proxies
85
86
If you specify a `proxy` option, then the request (and any subsequent
87
redirects) will be sent via a connection to the proxy server.
88
89
If your endpoint is an `https` url, and you are using a proxy, then
90
request will send a `CONNECT` request to the proxy server *first*, and
91
then use the supplied connection to connect to the endpoint.
92
93
That is, first it will make a request like:
94
95
```
96
HTTP/1.1 CONNECT endpoint-server.com:80
97
Host: proxy-server.com
98
User-Agent: whatever user agent you specify
99
```
100
101
and then the proxy server make a TCP connection to `endpoint-server`
102
on port `80`, and return a response that looks like:
103
104
```
105
HTTP/1.1 200 OK
106
```
107
108
At this point, the connection is left open, and the client is
109
communicating directly with the `endpoint-server.com` machine.
110
111
See [the wikipedia page on HTTP Tunneling](http://en.wikipedia.org/wiki/HTTP_tunnel)
112
for more information.
113
114
By default, when proxying `http` traffic, request will simply make a
115
standard proxied `http` request. This is done by making the `url`
116
section of the initial line of the request a fully qualified url to
117
the endpoint.
118
119
For example, it will make a single request that looks like:
120
121
```
122
HTTP/1.1 GET http://endpoint-server.com/some-url
123
Host: proxy-server.com
124
Other-Headers: all go here
125
126
request body or whatever
127
```
128
129
Because a pure "http over http" tunnel offers no additional security
130
or other features, it is generally simpler to go with a
131
straightforward HTTP proxy in this case. However, if you would like
132
to force a tunneling proxy, you may set the `tunnel` option to `true`.
133
134
If you are using a tunneling proxy, you may set the
135
`proxyHeaderWhiteList` to share certain headers with the proxy.
136
137
By default, this set is:
138
139
```
140
accept
141
accept-charset
142
accept-encoding
143
accept-language
144
accept-ranges
145
cache-control
146
content-encoding
147
content-language
148
content-length
149
content-location
150
content-md5
151
content-range
152
content-type
153
connection
154
date
155
expect
156
max-forwards
157
pragma
158
proxy-authorization
159
referer
160
te
161
transfer-encoding
162
user-agent
163
via
164
```
165
166
Note that, when using a tunneling proxy, the `proxy-authorization`
167
header is *never* sent to the endpoint server, but only to the proxy
168
server. All other headers are sent as-is over the established
169
connection.
170
171
### Controlling proxy behaviour using environment variables
172
173
The following environment variables are respected by `request`:
174
175
* `HTTP_PROXY` / `http_proxy`
176
* `HTTPS_PROXY` / `https_proxy`
177
* `NO_PROXY` / `no_proxy`
178
179
When `HTTP_PROXY` / `http_proxy` are set, they will be used to proxy non-SSL requests that do not have an explicit `proxy` configuration option present. Similarly, `HTTPS_PROXY` / `https_proxy` will be respected for SSL requests that do not have an explicit `proxy` configuration option. It is valid to define a proxy in one of the environment variables, but then override it for a specific request, using the `proxy` configuration option. Furthermore, the `proxy` configuration option can be explicitly set to false / null to opt out of proxying altogether for that request.
180
181
`request` is also aware of the `NO_PROXY`/`no_proxy` environment variables. These variables provide a granular way to opt out of proxying, on a per-host basis. It should contain a comma separated list of hosts to opt out of proxying. It is also possible to opt of proxying when a particular destination port is used. Finally, the variable may be set to `*` to opt out of the implicit proxy configuration of the other environment variables.
182
183
Here's some examples of valid `no_proxy` values:
184
185
* `google.com` - don't proxy HTTP/HTTPS requests to Google.
186
* `google.com:443` - don't proxy HTTPS requests to Google, but *do* proxy HTTP requests to Google.
187
* `google.com:443, yahoo.com:80` - don't proxy HTTPS requests to Google, and don't proxy HTTP requests to Yahoo!
188
* `*` - ignore `https_proxy`/`http_proxy` environment variables altogether.
189
190
## UNIX Socket
191
192
`request` supports the `unix://` protocol for all requests. The path is assumed to be absolute to the root of the host file system.
193
194
HTTP paths are extracted from the supplied URL by testing each level of the full URL against net.connect for a socket response.
195
196
Thus the following request will GET `/httppath` from the HTTP server listening on `/tmp/unix.socket`
197
198
```javascript
199
request.get('unix://tmp/unix.socket/httppath')
200
```
201
202
## Forms
203
204
`request` supports `application/x-www-form-urlencoded` and `multipart/form-data` form uploads. For `multipart/related` refer to the `multipart` API.
205
206
URL-encoded forms are simple.
207
208
```javascript
209
request.post('http://service.com/upload', {form:{key:'value'}})
210
// or
211
request.post('http://service.com/upload').form({key:'value'})
212
```
213
214
For `multipart/form-data` we use the [form-data](https://github.com/felixge/node-form-data) library by [@felixge](https://github.com/felixge). For the most basic case, you can pass your upload form data via the `formData` option.
215
216
217
```javascript
218
var formData = {
219
my_field: 'my_value',
220
my_buffer: new Buffer([1, 2, 3]),
221
my_file: fs.createReadStream(__dirname + '/unicycle.jpg'),
222
remote_file: request(remoteFile)
223
};
224
request.post({url:'http://service.com/upload', formData: formData}, function optionalCallback(err, httpResponse, body) {
225
if (err) {
226
return console.error('upload failed:', err);
227
}
228
console.log('Upload successful! Server responded with:', body);
229
});
230
```
231
232
For more advanced cases (like appending form data options) you'll need access to the form itself.
233
234
```javascript
235
var r = request.post('http://service.com/upload', function optionalCallback(err, httpResponse, body) {
236
if (err) {
237
return console.error('upload failed:', err);
238
}
239
console.log('Upload successful! Server responded with:', body);
240
})
241
242
// Just like always, `r` is a writable stream, and can be used as such (you have until nextTick to pipe it, etc.)
243
// Alternatively, you can provide a callback (that's what this example does — see `optionalCallback` above).
244
var form = r.form();
245
form.append('my_field', 'my_value');
246
form.append('my_buffer', new Buffer([1, 2, 3]));
247
form.append('my_buffer', fs.createReadStream(__dirname + '/unicycle.jpg'), {filename: 'unicycle.jpg'});
248
```
249
See the [form-data](https://github.com/felixge/node-form-data) README for more information & examples.
250
251
Some variations in different HTTP implementations require a newline/CRLF before, after, or both before and after the boundary of a `multipart/form-data` request. This has been observed in the .NET WebAPI version 4.0. You can turn on a boundary preambleCRLF or postamble by passing them as `true` to your request options.
252
253
```javascript
254
request(
255
{ method: 'PUT'
256
, preambleCRLF: true
257
, postambleCRLF: true
258
, uri: 'http://service.com/upload'
259
, multipart:
260
[ { 'content-type': 'application/json'
261
, body: JSON.stringify({foo: 'bar', _attachments: {'message.txt': {follows: true, length: 18, 'content_type': 'text/plain' }}})
262
}
263
, { body: 'I am an attachment' }
264
]
265
}
266
, function (error, response, body) {
267
if (err) {
268
return console.error('upload failed:', err);
269
}
270
console.log('Upload successful! Server responded with:', body);
271
}
272
)
273
```
274
275
276
## HTTP Authentication
277
278
```javascript
279
request.get('http://some.server.com/').auth('username', 'password', false);
280
// or
281
request.get('http://some.server.com/', {
282
'auth': {
283
'user': 'username',
284
'pass': 'password',
285
'sendImmediately': false
286
}
287
});
288
// or
289
request.get('http://some.server.com/').auth(null, null, true, 'bearerToken');
290
// or
291
request.get('http://some.server.com/', {
292
'auth': {
293
'bearer': 'bearerToken'
294
}
295
});
296
```
297
298
If passed as an option, `auth` should be a hash containing values `user` || `username`, `pass` || `password`, and `sendImmediately` (optional). The method form takes parameters `auth(username, password, sendImmediately)`.
299
300
`sendImmediately` defaults to `true`, which causes a basic authentication header to be sent. If `sendImmediately` is `false`, then `request` will retry with a proper authentication header after receiving a `401` response from the server (which must contain a `WWW-Authenticate` header indicating the required authentication method).
301
302
Note that you can also use for basic authentication a trick using the URL itself, as specified in [RFC 1738](http://www.ietf.org/rfc/rfc1738.txt).
303
Simply pass the `user:password` before the host with an `@` sign.
304
305
```javascript
306
var username = 'username',
307
password = 'password',
308
url = 'http://' + username + ':' + password + '@some.server.com';
309
310
request({url: url}, function (error, response, body) {
311
// Do more stuff with 'body' here
312
});
313
```
314
315
Digest authentication is supported, but it only works with `sendImmediately` set to `false`; otherwise `request` will send basic authentication on the initial request, which will probably cause the request to fail.
316
317
Bearer authentication is supported, and is activated when the `bearer` value is available. The value may be either a `String` or a `Function` returning a `String`. Using a function to supply the bearer token is particularly useful if used in conjuction with `defaults` to allow a single function to supply the last known token at the time or sending a request or to compute one on the fly.
318
319
## OAuth Signing
320
321
```javascript
322
// Twitter OAuth
323
var qs = require('querystring')
324
, oauth =
325
{ callback: 'http://mysite.com/callback/'
326
, consumer_key: CONSUMER_KEY
327
, consumer_secret: CONSUMER_SECRET
328
}
329
, url = 'https://api.twitter.com/oauth/request_token'
330
;
331
request.post({url:url, oauth:oauth}, function (e, r, body) {
332
// Ideally, you would take the body in the response
333
// and construct a URL that a user clicks on (like a sign in button).
334
// The verifier is only available in the response after a user has
335
// verified with twitter that they are authorizing your app.
336
var access_token = qs.parse(body)
337
, oauth =
338
{ consumer_key: CONSUMER_KEY
339
, consumer_secret: CONSUMER_SECRET
340
, token: access_token.oauth_token
341
, verifier: access_token.oauth_verifier
342
}
343
, url = 'https://api.twitter.com/oauth/access_token'
344
;
345
request.post({url:url, oauth:oauth}, function (e, r, body) {
346
var perm_token = qs.parse(body)
347
, oauth =
348
{ consumer_key: CONSUMER_KEY
349
, consumer_secret: CONSUMER_SECRET
350
, token: perm_token.oauth_token
351
, token_secret: perm_token.oauth_token_secret
352
}
353
, url = 'https://api.twitter.com/1.1/users/show.json?'
354
, params =
355
{ screen_name: perm_token.screen_name
356
, user_id: perm_token.user_id
357
}
358
;
359
url += qs.stringify(params)
360
request.get({url:url, oauth:oauth, json:true}, function (e, r, user) {
361
console.log(user)
362
})
363
})
364
})
365
```
366
367
## Custom HTTP Headers
368
369
HTTP Headers, such as `User-Agent`, can be set in the `options` object.
370
In the example below, we call the github API to find out the number
371
of stars and forks for the request repository. This requires a
372
custom `User-Agent` header as well as https.
373
374
```javascript
375
var request = require('request');
376
377
var options = {
378
url: 'https://api.github.com/repos/mikeal/request',
379
headers: {
380
'User-Agent': 'request'
381
}
382
};
383
384
function callback(error, response, body) {
385
if (!error && response.statusCode == 200) {
386
var info = JSON.parse(body);
387
console.log(info.stargazers_count + " Stars");
388
console.log(info.forks_count + " Forks");
389
}
390
}
391
392
request(options, callback);
393
```
394
395
## request(options, callback)
396
397
The first argument can be either a `url` or an `options` object. The only required option is `uri`; all others are optional.
398
399
* `uri` || `url` - fully qualified uri or a parsed url object from `url.parse()`
400
* `qs` - object containing querystring values to be appended to the `uri`
401
* `method` - http method (default: `"GET"`)
402
* `headers` - http headers (default: `{}`)
403
* `body` - entity body for PATCH, POST and PUT requests. Must be a `Buffer` or `String`.
404
* `form` - when passed an object or a querystring, this sets `body` to a querystring representation of value, and adds `Content-type: application/x-www-form-urlencoded; charset=utf-8` header. When passed no options, a `FormData` instance is returned (and is piped to request).
405
* `auth` - A hash containing values `user` || `username`, `pass` || `password`, and `sendImmediately` (optional). See documentation above.
406
* `json` - sets `body` but to JSON representation of value and adds `Content-type: application/json` header. Additionally, parses the response body as JSON.
407
* `multipart` - (experimental) array of objects which contains their own headers and `body` attribute. Sends `multipart/related` request. See example below.
408
* `preambleCRLF` - append a newline/CRLF before the boundary of your `multipart/form-data` request.
409
* `postambleCRLF` - append a newline/CRLF at the end of the boundary of your `multipart/form-data` request.
410
* `followRedirect` - follow HTTP 3xx responses as redirects (default: `true`). This property can also be implemented as function which gets `response` object as a single argument and should return `true` if redirects should continue or `false` otherwise.
411
* `followAllRedirects` - follow non-GET HTTP 3xx responses as redirects (default: `false`)
412
* `maxRedirects` - the maximum number of redirects to follow (default: `10`)
413
* `encoding` - Encoding to be used on `setEncoding` of response data. If `null`, the `body` is returned as a `Buffer`. Anything else **(including the default value of `undefined`)** will be passed as the [encoding](http://nodejs.org/api/buffer.html#buffer_buffer) parameter to `toString()` (meaning this is effectively `utf8` by default).
414
* `pool` - A hash object containing the agents for these requests. If omitted, the request will use the global pool (which is set to node's default `maxSockets`)
415
* `pool.maxSockets` - Integer containing the maximum amount of sockets in the pool.
416
* `timeout` - Integer containing the number of milliseconds to wait for a request to respond before aborting the request
417
* `proxy` - An HTTP proxy to be used. Supports proxy Auth with Basic Auth, identical to support for the `url` parameter (by embedding the auth info in the `uri`)
418
* `oauth` - Options for OAuth HMAC-SHA1 signing. See documentation above.
419
* `hawk` - Options for [Hawk signing](https://github.com/hueniverse/hawk). The `credentials` key must contain the necessary signing info, [see hawk docs for details](https://github.com/hueniverse/hawk#usage-example).
420
* `strictSSL` - If `true`, requires SSL certificates be valid. **Note:** to use your own certificate authority, you need to specify an agent that was created with that CA as an option.
421
* `jar` - If `true` and `tough-cookie` is installed, remember cookies for future use (or define your custom cookie jar; see examples section)
422
* `aws` - `object` containing AWS signing information. Should have the properties `key`, `secret`. Also requires the property `bucket`, unless you’re specifying your `bucket` as part of the path, or the request doesn’t use a bucket (i.e. GET Services)
423
* `httpSignature` - Options for the [HTTP Signature Scheme](https://github.com/joyent/node-http-signature/blob/master/http_signing.md) using [Joyent's library](https://github.com/joyent/node-http-signature). The `keyId` and `key` properties must be specified. See the docs for other options.
424
* `localAddress` - Local interface to bind for network connections.
425
* `gzip` - If `true`, add an `Accept-Encoding` header to request compressed content encodings from the server (if not already present) and decode supported content encodings in the response. **Note:** Automatic decoding of the response content is performed on the body data returned through `request` (both through the `request` stream and passed to the callback function) but is not performed on the `response` stream (available from the `response` event) which is the unmodified `http.IncomingMessage` object which may contain compressed data. See example below.
426
* `tunnel` - If `true`, then *always* use a tunneling proxy. If
427
`false` (default), then tunneling will only be used if the
428
destination is `https`, or if a previous request in the redirect
429
chain used a tunneling proxy.
430
* `proxyHeaderWhiteList` - A whitelist of headers to send to a
431
tunneling proxy.
432
433
434
The callback argument gets 3 arguments:
435
436
1. An `error` when applicable (usually from [`http.ClientRequest`](http://nodejs.org/api/http.html#http_class_http_clientrequest) object)
437
2. An [`http.IncomingMessage`](http://nodejs.org/api/http.html#http_http_incomingmessage) object
438
3. The third is the `response` body (`String` or `Buffer`, or JSON object if the `json` option is supplied)
439
440
## Convenience methods
441
442
There are also shorthand methods for different HTTP METHODs and some other conveniences.
443
444
### request.defaults(options)
445
446
This method returns a wrapper around the normal request API that defaults to whatever options you pass in to it.
447
448
**Note:** You can call `.defaults()` on the wrapper that is returned from `request.defaults` to add/override defaults that were previously defaulted.
449
450
For example:
451
```javascript
452
//requests using baseRequest() will set the 'x-token' header
453
var baseRequest = request.defaults({
454
headers: {x-token: 'my-token'}
455
})
456
457
//requests using specialRequest() will include the 'x-token' header set in
458
//baseRequest and will also include the 'special' header
459
var specialRequest = baseRequest.defaults({
460
headers: {special: 'special value'}
461
})
462
```
463
464
### request.put
465
466
Same as `request()`, but defaults to `method: "PUT"`.
467
468
```javascript
469
request.put(url)
470
```
471
472
### request.patch
473
474
Same as `request()`, but defaults to `method: "PATCH"`.
475
476
```javascript
477
request.patch(url)
478
```
479
480
### request.post
481
482
Same as `request()`, but defaults to `method: "POST"`.
483
484
```javascript
485
request.post(url)
486
```
487
488
### request.head
489
490
Same as request() but defaults to `method: "HEAD"`.
491
492
```javascript
493
request.head(url)
494
```
495
496
### request.del
497
498
Same as `request()`, but defaults to `method: "DELETE"`.
499
500
```javascript
501
request.del(url)
502
```
503
504
### request.get
505
506
Same as `request()` (for uniformity).
507
508
```javascript
509
request.get(url)
510
```
511
### request.cookie
512
513
Function that creates a new cookie.
514
515
```javascript
516
request.cookie('cookie_string_here')
517
```
518
### request.jar
519
520
Function that creates a new cookie jar.
521
522
```javascript
523
request.jar()
524
```
525
526
527
## Examples:
528
529
```javascript
530
var request = require('request')
531
, rand = Math.floor(Math.random()*100000000).toString()
532
;
533
request(
534
{ method: 'PUT'
535
, uri: 'http://mikeal.iriscouch.com/testjs/' + rand
536
, multipart:
537
[ { 'content-type': 'application/json'
538
, body: JSON.stringify({foo: 'bar', _attachments: {'message.txt': {follows: true, length: 18, 'content_type': 'text/plain' }}})
539
}
540
, { body: 'I am an attachment' }
541
]
542
}
543
, function (error, response, body) {
544
if(response.statusCode == 201){
545
console.log('document saved as: http://mikeal.iriscouch.com/testjs/'+ rand)
546
} else {
547
console.log('error: '+ response.statusCode)
548
console.log(body)
549
}
550
}
551
)
552
```
553
554
For backwards-compatibility, response compression is not supported by default.
555
To accept gzip-compressed responses, set the `gzip` option to `true`. Note
556
that the body data passed through `request` is automatically decompressed
557
while the response object is unmodified and will contain compressed data if
558
the server sent a compressed response.
559
560
```javascript
561
var request = require('request')
562
request(
563
{ method: 'GET'
564
, uri: 'http://www.google.com'
565
, gzip: true
566
}
567
, function (error, response, body) {
568
// body is the decompressed response body
569
console.log('server encoded the data as: ' + (response.headers['content-encoding'] || 'identity'))
570
console.log('the decoded data is: ' + body)
571
}
572
).on('data', function(data) {
573
// decompressed data as it is received
574
console.log('decoded chunk: ' + data)
575
})
576
.on('response', function(response) {
577
// unmodified http.IncomingMessage object
578
response.on('data', function(data) {
579
// compressed data as it is received
580
console.log('received ' + data.length + ' bytes of compressed data')
581
})
582
})
583
```
584
585
Cookies are disabled by default (else, they would be used in subsequent requests). To enable cookies, set `jar` to `true` (either in `defaults` or `options`) and install `tough-cookie`.
586
587
```javascript
588
var request = request.defaults({jar: true})
589
request('http://www.google.com', function () {
590
request('http://images.google.com')
591
})
592
```
593
594
To use a custom cookie jar (instead of `request`’s global cookie jar), set `jar` to an instance of `request.jar()` (either in `defaults` or `options`)
595
596
```javascript
597
var j = request.jar()
598
var request = request.defaults({jar:j})
599
request('http://www.google.com', function () {
600
request('http://images.google.com')
601
})
602
```
603
604
OR
605
606
```javascript
607
// `npm install --save tough-cookie` before this works
608
var j = request.jar();
609
var cookie = request.cookie('key1=value1');
610
var url = 'http://www.google.com';
611
j.setCookieSync(cookie, url);
612
request({url: url, jar: j}, function () {
613
request('http://images.google.com')
614
})
615
```
616
617
To inspect your cookie jar after a request
618
619
```javascript
620
var j = request.jar()
621
request({url: 'http://www.google.com', jar: j}, function () {
622
var cookie_string = j.getCookieString(uri); // "key1=value1; key2=value2; ..."
623
var cookies = j.getCookies(uri);
624
// [{key: 'key1', value: 'value1', domain: "www.google.com", ...}, ...]
625
})
626
```
627
628
## Debugging
629
630
There are at least three ways to debug the operation of `request`:
631
632
1. Launch the node process like `NODE_DEBUG=request node script.js`
633
(`lib,request,otherlib` works too).
634
635
2. Set `require('request').debug = true` at any time (this does the same thing
636
as #1).
637
638
3. Use the [request-debug module](https://github.com/nylen/request-debug) to
639
view request and response headers and bodies.
640