- [Tim] All right, so we found out
at the end of the last video
that some methods implicitly perform the connectMethod step.
So let's move on now and try reading a webpage
that doesn't exist to see that
or see if we actually get the 404 response code back.
So what we're going to do is put a bogus page on line 14.
So we're going to make the URL as it was, example.org,
but we're gonna put /somepage.html instead.
So if we actually try running that,
you can see we got a response code of 404,
Error reading web page.
We've returned at that point, so we don't do any
more processing, and we don't try and retrieve or read
the contents of the page, which in this case, doesn't exist.
And incidentally, what we can do is call
the HttpURLConnection.getResponseMessage method.
We can get more information about the error.
So we can change that and put
on the next line connection.getResponseMessage.
So let's do that and see what it outputs for us.
You can see there it's come back and said not found.
Not very exciting, in this case,
but depending on the response, you might get information
that will help you figure out actually what went wrong.
Now, if you were doing a post,
you'd call the connection.geOutputStream method to get
an output stream which you'd use to write to the connection.
And you can also wrap the output stream
in DataOutputStream if you want to.
You're gonna see some sample post code in a minute,
but let's actually try passing parameters as part
of the URL, which is something you'll often wanna do.
Now, we're gonna use the Flickr API service to fetch
the most recently uploaded photos tagged with cats.
Now, if I click on this link, go to this link in the browser
first, let's just take a look at what that looks like.
So there's the link.
And looking at that, you can see that we can use
this URL here on the page,
and we can pass various parameters to that.
So the query parameters, we can include
to specify which photos you'd like returned.
So what we wanna do is perhaps use this tags option here,
a comma delimited list of tags to filter the feed by.
So we could take a copy of that first,
paste it into a new link, and then we'll put ?tags=cats,
just to confirm that actually works,
and you can see that we've got a feed there,
an XML feed of various photos.
And when you go to do this, this is updated in real time,
so you'll be getting different content
than you can see in my particular case here.
And what I've done here it always a good idea
when you're testing.
Try out the URL that you're working with using a browser
before using it in the code.
That way, you can see for sure that you've got the URL
correct, and its functioning and it's returning
the right data instead of, you know, just pasting it
directly into your code and then not knowing if something
doesn't work whether it's the URL at fault
or something you've done wrong in your code.
But in our case, you can see that that's working nicely,
and you can see under the title for HTML,
recently uploaded tagged cats,
and you can see this first entry here.
Entry, the first bit of HTML tag there for entry.
And we see that there's a link there as well.
If we scroll down a little bit further,
there's a link there, as you can see.
So I could take a copy of that link if I wanted to,
paste it in, and have a look at it,
and there's a picture of a cat, as you can see.
So you'll be able to do something similar
if you wanted to with the code
that you'll get back from using this particular URL.
And incidentally, in case you're wondering why it's updated,
this is actually a recent list, as the title says.
Recently uploaded or recent uploads
of photos that are tagged cats.
So because Flickr is an image site that retrieves
or receives tens of thousands of images a day,
you'll find that when you go to run this,
you'll be getting different images showing there.
So you can see that there's quite a few entries here.
So that's the first entry.
There's another one.
There's another one,
and another one, and I'm not gonna do any more,
but it's quite a few there, as you can see.
Now, if you were to read this URL into an application,
you could display the photos or display a list of links
or literally whatever else you wanna do
within your application.
And actually, within my Android course,
I actually do just that.
We do a similar thing, and in the course,
we actually create an app that actually
extracts the contents out of this XML,
which you can see on the screen there,
and enables you to display the photo on an Android device.
So you can see that we've got the information.
You could actually use that to do
whatever you'd wanna do within your application.
And if you were going to do that, as we've done
in the Android app that is in my Android course,
you'd use XML or an XML/RSS library
to read the feed rather than writing code to parse it.
No point reinventing the wheel when there's code out there
to do that for you automatically.
All right, so now you know that the URL works
and takes you to something valid,
what we can do is write the code to get the most recently
uploaded photos tagged with cats at Flickr.
So all we really need to do here is change that URL
that showed there and add a query to that,
?tags=cats to actually get the specific data relating,
in this case, to recent photos
that are tagged with the word cats.
So I go back to the code again there now.
All right, so what I'm gonna do now is change
this example here, the code on line 14.
Let's go ahead and go back to the browser.
We're gonna copy that URL.
We're gonna paste that in there,
and then what we can do is, on the end of that,
we can add ?tags=cats as we've done in the browser.
We'll open the connection.
We'll leave the get in there.
We'll leave the request property
for the user agent as well as the timeout.
and you should be able to leave everything else
as it is now and run this, and we should be able to see
the XML that's returned from that link.
So let's actually try running that.
And you can see there that we've got the XML data
that's come back from that link.
You can see there's quite a bit there, and right up
to the top there, you can see we got a response code of 200.
And of course we could change that as well.
Instead of cats, dogs.
Run the same programme again.
Because Flickr is updating that feed in real time,
we'll get some data back relating to dogs
in this particular case.
So if I go back up to the top,
you can see recent uploads tagged dogs,
and you'd find, if we actually had a look
at some of these links, we'd find the data
would be be relating to dogs in this particular case.
Pretty easy, as you can see,
to extract data the way we have there.
Now, at the moment, we've got a URL that's HTTPS,
but it's actually using the HTTP URL connection,
and not the HTTPS URL connection.
So what's happening is that we're not really truly using
the HTTPS protocol.
Now, that's actually becoming more common,
so you'd need to use the HTTPS URL connection instance
to actually achieve that.
And out of interest, that connection class, the HTTPS
URL connection class, isn't in the java.net package.
It's in the the javax.net.ssl.
But we won't do that now,
but that is something you can actually do.
All right, before we move on,
let's take a quick look on the screen now and a slide
at how you go about writing tool connection.
Now, I'm not actually going to demonstrate this,
but the code will give you
a pretty good idea of how it's done.
All right, so you can see the code there.
We start off on line one, creating a new URL,
and then we create a HTTP URL connection.
We set the request method to post on line three,
and we set the user agent to Chrome on line four,
and we actually set on line five the content type as well.
So we're basically telling the server
the content type that we'll be sending
as well as the length of the content on line eight.
Now, you could pass these parameters are part of the URL,
but there are times when you wanna pass 'em
separately as we're doing here.
It gives you a few more options.
The parameter string is the part of the URL, by the way,
that appears after a question mark,
and you saw just previously when we ran our cats and dogs
Flickr application, I did a ?tags=dogs.
So that's the parameter that I'm talking about.
And in the case of the code on line seven and eight,
that's a way to actually send multiple parameters.
Firstly on line eight, we're sending the parameter's length,
but if you look down the code there down on line 15,
we're actually writing out the bytes
relating to the parameters there as well.
Now, there are other ways to write this code.
Depending on the server and the request,
you might have to set more properties
when you're configuring the connection.
But this should give you an example of how you'd
pass parameters separately from the URL.
Now, in this particular case, the URL is a PHP script,
which could be the action that's performed
when a user submits a form, for example.
And looking back to that code,
you'll see that on lines 11, we set the output to be true,
so that's where the system would then ultimately know
to write the code out, and then of course we're using
the output.writeBytes on line 15 to start writing
some output, and that's wrapped around the data output
stream, as you can see, because output.writeBytes
is obviously using the data output stream itself.
All right, so at this point, you've seen how to use
the HttpUrlConnection and UrlConnection classes
to read webpages and retrieve specific data from webpages
by passing parameters, and that's actually great.
But here's the thing to keep in mind:
many Java developers don't use the classes
in the java.net package when they wanna work with URLs.
So the java.net package has been in the JDK
all the way back since Java 1.0, and unfortunately,
it really hasn't changed much since then.
Under the covers,
it's just an older version of the HTTP protocol.
So classes that use the newer HTTP/2.0 protocol
were proposed and worked on for Java 9, and the proposal
listed the following problems, as you can see on the screen,
and I've taken this from the link you can see
on the screen there, open.jdk.java.net/jeps/110.
So the problems that they found were the base UrlConnection
API was designed with multiple protocols in mind,
nearly all of which are now defunct,
and that's ftp, gopher, et cetera.
The API predates HTTP/1.1 and is too abstract as a result.
It's hard to use, with many undocumented behaviours,
and it works in blocking modes only;
that is, one thread per request or response,
and most importantly, it's very hard to maintain.
Unfortunately, the behaviour of the java.net
HTTP classes can be frustrating.
Try sending parameters using the POST method,
as in the sample code that I showed on the screen,
and you could be tearing your hair out.
That's because it doesn't always work,
and when developers go to the web and try to find out why,
other developers will usually say words to the effect of,
don't use the java.net classes.
Use a third-party library.
Now, unfortunately, the proposed Java 9 API has been
placed in what's called the incubator, meaning that,
based on the feedback from developers who tried the early
access Java 9 features, it's not ready for primetime.
In other words, the API won't be in the base Java 9 JDK
but developers will still be able to try it out
by adding it to their projects.
Now, based on the feedback the API team
expects to get from even more developers,
the API may well be revised and then added to the JDK
or it may also be deleted.
So it's not a good idea to use a new API in any
applications you plan to release to users in the future.
But in the meantime, many developers are using third-party
libraries when they want to work with a HTTP protocol.
So two popular ones I wanna talk about briefly are jetty,
and that's available from www.eclipse.org/jetty,
and as well as the Apache HTTPClient,
hc.apache.org/httpcomponenets-client-ga.
Keep in mind that even though the Apache library
only supports HTTP/1.1, overall, developers like it,
because it's much easier to use than the java.net classes.
So, it's important that you got a taste
of the high-level API though in the java.net classes
so that you can read code that uses them,
but when you wanna work with the HTTP protocol,
you might wanna consider using jetty
or the Apache HTTPClient libraries.
Now, to give you a taste of the Apache HTTPClient library,
let's use it to read the example site of http://example.org.
So what I'm going to do is download
what is the current latest release.
So let's go to the page and do that.
There's the link.
So I'm going to download the latest version
of the Apache HTTPClient library, and you can see
at the time that I'm doing this, the version's 4.5.3,
but if the versions change when you go to do it,
just download the latest version.
I'm just going to download the zip.
It doesn't really matter which one you download.
And I'm just going to put it on my desktop.
Save.
You can see, it's only a small file,
and I'm going to go to that location.
I'm going to unzip it.
All right, now that I'm unzipped it,
I'm going to go back to IntelliJ,
and I go back and select File > Project Structure,
and we're going to select Libraries,
and we're going to add that.
Click on the plus to add it as a library.
Click on Java, and I'm going to navigate to that folder,
which in my case, was the desktop.
Go into the lib folder, and I'm going to select
all the jars that were part of that.
It's quite a few there, as you can see.
Click on Open.
We'll add it to the selected module, which in this case,
is Networking, for this particular project.
You can see all the files are there now,
and we click on OK.
IntelliJ's indexed those briefly,
and we've now added them to the project.
So what I'm going to do now is create a new class,
and I'll call this class ApacheTest.
And I'm going to create a public method,
public main method here, psvm, tab,
public static void main, and let's actually see
what the equivalent code is to retrieve the webpage.
Instead of using java.net,
we're going to be using the Apache HTTPClient.
So we've added that as an option here.
We've created a new class, ApacheTest,
and I've created a new main method.
So the actual code to do the equivalent,
we're going to start with CloseableHttpClient
httpClient = HttpClients.createDefault;
HttpGet request = new HttpGet,
and we're gonna pass the website.
In this case, it's going to be http://example.org.
Next, we're going to add a header.
So request.addHeader, and that's going to be the User-Agent.
We'll go with Chrome again, remembering you don't actually
have to be running on Chrome or anything like that.
But we're just passing something there.
Lets execute this response, and we're gonna return
or get returned to us a CloseableHttpResponse.
We'll call that response,
and it's equal to httpClient.execute, and our request.
Then we need to add a try, and what we'll actually do
is we'll put that CloseableHttpResponse in the try,
'cause it really needs to catch, as you can see there,
it wants to catch an IOException.
Then we're gonna start by printing out the response code.
So response code = +,
and it's response.getStatusLine().getStatusCode();
then we're gonna do BufferedReader inputReader,
and we can actually probably copy a lot of this
from the other one, so let's actually do that.
I'm gonna copy that line, those lines of code,
and we'll paste that in here,
and we'll paste that over the top of the BufferedReader.
Then we'll just make a few changes to the code there,
and let's close down the run window.
Let's also catch the IOException, just to remove that
as an error in the code.
IOException e.
We'll just print out IOException = + e.getMessage();
and in terms of the code, we've got BufferedReader
inputReader = new, BufferedReader,
it's gonna be new InputStreamReader,
but it's not going to be connection this time.
It's going to be a bit different there.
It's going to be...
I'll just delete that out.
It's going to be response.getEntity().getContent().
Okay, then we've got the string line there.
while line is not equal to inputReader.readLine,
not equal to null,
and we're outputting the content retrieved,
ending the while loop, and then closing at the end there.
The other thing we'll do down here,
we'll add a finally.
We're gonna do a response.close, and then we're
just gonna make a bit of a change here.
There, and we can assign that variable there,
outside of the try, and then we'll actually
just update the variable that way.
And therefore, we're going to be able
to actually close it there.
And we'll just catch an IOException there.
Because we're not using a try-with-resources there,
we just need to make sure that we can close it
successfully in case it wasn't initialised.
So there's the code using the Apache HTTPClient.
So you can see that it's similar to what we've had before,
except that we're creating this CloseableHttpClient instance
on line 18, and then we're using or we're getting
a HttpGet instance for the URL again on line 19.
Now, we configure the request object, not the HttpClient,
so it's a little bit different.
Once we've configured it, then all we're doing here again
is just setting the User-Agent on line 20.
We're then calling the CloseableHttpClient.execute
method on line 24.
That sends off the request.
Now, when using the CloseableHttpResponse,
it is highly recommended that you call the close method
in a finally block as I've done down here.
The reason is that otherwise, system resources
may not be released correctly.
The other way I could have done this, by the way,
I could have actually changed this
to the throws IOException here,
which is something that you'll see is done regularly,
and by doing that, I could have actually had this in here
outside of the try loop.
Response.
Remove that altogether, I can then remove this final try
and catch and clean it up that way,
because we've thrown the IOException at this level.
So that's another way of doing it
if you didn't want to handle the extra call
outside of the try as I did there.
Makes the finally look a little bit more tidy
because we're throwing the IOException from the method
instead of trying to catch that in the finally block.
But in any event, once we've got the response
we get the response code by calling the getStatus line.
You can see that here on line 25, and then getting from that
the StatusCode call on the result.
And the rest of the code then you saw
was pretty much a copy and paste.
The only real change was this
response.getEntity().getContent for the actual method.
Now, there are other ways of handling the content,
but we're doing it this way because it's very similar
to how we handle the response
when using the java.net HttpURLConnection class.
So let's actually just run this,
making sure that it actually works.
Run ApacheTest.main.
And you can see we're able to retrieve
our example.org page with no problems,
so we got our response code correctly of 200.
So that's a taste of one of the popular
third-party libraries and also that's the end
of how to work with a high-level API and the end
of this section on the Java network programming.
I hope you got a lot out of it.
Thanks very much for watching, and let's move on now.