Python urllib2 file open

The return value from urlopen gives access to the headers from the HTTP server through the info method, and the data for the remote resource via methods like read and readlines. The file-like object returned by urlopen is iterable:. Arguments can be passed to the server by encoding them with urllib.

For more precise control, you may want to instantiate and use a Request object directly. As the examples above illustrate, the default User-agent header value is made up of the constant Python-urllib , followed by the Python interpreter version. Using a custom agent also allows them to control crawlers using a robots. The last line of the output shows our custom value. The following are 30 code examples for showing how to use urllib2.

These examples are extracted from open source projects. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar. More from urllib2. Request re. Project: yabgp Author: smartbgp File: hijack. Project: ccs-calendarserver Author: apple File: population.

Project: d4rkc0de Author: knightmare File: linksysbrute. Request sys. Project: d4rkc0de Author: knightmare File: webauthbrute. HTTPError, httplib. Project: incubator-sdap-nexus Author: apache File: wls. Project: darkc0de-old-stuff Author: tuwid File: linksysbrute. Return values and exceptions raised should be the same as those of urlopen. This method is also not defined in BaseHandler , but will be called, if it exists, on an instance of a subclass, when an HTTP error with code nnn occurs.

This method is not defined in BaseHandler , but subclasses should define it if they want to pre-process requests of the given protocol. The return value should be a Request object. This method is not defined in BaseHandler , but subclasses should define it if they want to post-process responses of the given protocol. The return value should implement the same interface as the return value of urlopen.

See RFC for details of the precise meanings of the various redirection codes. Return a Request or None in response to a redirect. The default implementation of this method does not strictly follow RFC , which says that and responses to POST requests must not be automatically redirected without confirmation by the user.

In reality, browsers do allow automatic redirection of these responses, changing the POST to a GET , and the default implementation reproduces this behavior. The cookielib. CookieJar in which cookies are stored.

The method will modify requests to go through the proxy, by calling request. This causes user, passwd to be used as authentication tokens when authentication for realm and a super-URI of any of the given URIs is given.

In either case, the authority must not contain a userinfo component so, "python. Open the file locally, if there is no host name, or the host name is 'localhost'. Change the protocol to ftp otherwise, and retry opening it using parent. Open the FTP file indicated by req. The login is always done with empty username and password. Raise a URLError exception. Eventually, urllib2. Here we are sending a data-stream to the stdin of a CGI and reading the data it returns to us.

Note that this example will only work when the Python installation supports SSL. This example replaces the default ProxyHandler with one that uses programmatically-supplied proxy URLs, and adds proxy authorization support with ProxyBasicAuthHandler.

Use the headers argument to the Request constructor, or:. OpenerDirector automatically adds a User-Agent header to every Request. To change this:. Enter search terms or a module, class or function name. Navigation index modules next previous Python 2. Internet Protocols and Support ».