So far,
we tested every single visible feature within this website.
And we clicked on every single, visible button.
And as a result
we discovered a lot of bugs and vulnerabilities.
We also discovered bugs and vulnerabilities
in hidden features and in hidden code.
And we were able to do that
by relying on the filter functionality in here.
So, we went to the target, we went to the filter,
and based on some clues that we saw on the website,
we searched for certain parameters,
such as the F parameter in here, cuz we got a clue for it.
While we were testing one of the pages,
while we were testing the menu page.
And as a result, we discovered other values
that we can send to this parameter
such as the value that we sent in here
to generate credentials
which allowed us to get the credentials.
And then we logged into the admin,
and then we were able to test even more features.
I'm saying all of this because I'm trying to tell you
that testing every single visible feature,
and press-in every single button is not enough
to properly cover the web application,
because as you can see,
we were able to discover bugs and vulnerabilities,
in hidden code and in hidden features,
therefore, usually at the end of the pen test,
or the bug hunt,
I would dedicate some time and put some effort,
and to trying to discover hidden features,
and hidden parameters, and then test them,
and hopefully, discover more bugs and vulnerabilities.
So in this lecture,
I'm gonna show you how to use regular expressions
in order to find more endpoints in this website
that we were not able to find
by simply clicking on the buttons so that hopefully,
we can discover bugs and vulnerabilities in these endpoints.
We're gonna do this using regular expressions, like I said,
and basically using regular expressions,
we can tell burp,
to search for a specific patterns
within all of the files that it has intercepted.
So, within all of the JavaScript,
and within all of the application files that were exchanged
between the browser and the web server,
and as a result,
hopefully, we will be able to find endpoints.
Now, regular expressions are used all over the industry
not even in cyber security only,
it's used in programming,
and it's used in so many other applications,
because they're very powerful,
because they allow us to specify a pattern,
and then get the program to match a string, based on that.
I know, all sounds confusing,
but let me show you how to use them in real life
to discover endpoints,
and you'll see how easy it is.
So basically, the goal is, I wanna see any other parameters
that I can give values to and manipulate
in order to discover bugs and vulnerabilities.
So we learned before, even at the start of the course,
we always look for something like a question mark,
followed by some text,
followed by an equal sign.
So, what I wanna do basically,
I wanna tell burp,
to find any sequence of characters,
that matches this pattern.
I want a question mark,
followed by some text in here,
so let's just say X, X, X, X, X,
and followed by an equal sign.
I wanted to search all of the files
that it intercepted in the target website.
And if it find any of them,
I wanted to show it to me,
so that I can test these parameters
for more bugs and vulnerabilities.
And we can do that very easily using regular expressions.
So first of all, I'm just gonna show you,
we're gonna go to the target.
We're gonna click on the filter,
and we're gonna enable the Regex, text box in here.
So, we're telling burp,
I'm not gonna search for a normal text,
I'm gonna search for regex.
And like I said, we wanted to start with a question mark.
So, we're gonna put a question mark,
and then we wanna match as many characters as possible,
followed by an equal sign.
And here's where you're gonna see the power of regex.
We're gonna tell burp,
to match as many characters as possible
between a question mark,
and between an equal sign by basically saying, dot star.
So, dot means that I want to match any type of character,
and star means any number of characters.
So if I just put one dot,
it's gonna match only one character,
but I don't care how many characters are there.
As long as they are enclosed
between a question mark and an equal sign.
Now, as you can see,
in regex, you can use dots and stars,
so you can use special characters
to indicate different meanings.
So the question mark can actually be used
to stop matching before the end of the line.
So, to stop matching once this pattern is found,
but we actually don't want to say that in here.
We want this question mark
to be part of the regular expression.
Like we said, because we're looking for a question mark,
followed by a series of characters,
followed by an equal.
And therefore, to tell burp,
to not treat this question mark as a special character,
and to actually treat it as part of the pattern,
I'm gonna have to put a backward slash before it.
And finally, we wanna tell burp,
to not continue matching,
because remember, it's actually gonna search for this
in code, and there could be a lot of lines of code.
So, I wanted to stop looking for this pattern.
Once this pattern is satisfied,
I don't want it to continue matching all the way
up to the end of the line.
And that's why I'm actually gonna use a question mark
in here.
So basically, just to make this simpler,
we're using a dot, star to say, I wanna search
for any sequence of characters,
for any pattern that matches this,
as long as it starts with a question mark,
and ends with an equal sign,
which is exactly, what we want,
when we are looking for parameters
that we can manipulate to discover bugs.
And then we're using the backward slash to say,
this question mark should not be treated
as a special character,
it's actually part of the pattern.
And we're actually using a special character
as a question mark in here to say,
that I want to stop as soon as this pattern is discovered,
I don't want you to continue matching the whole line
that contains this pattern.
Then you can modify the settings in here
if you wanna match any other files,
like CSS files, starlink files, and so on,
but I'm happy with what I have in here.
And once you're happy,
we're gonna click on apply, and perfect,
that should have filtered what I'm looking for.
So, as you can see, we have all of the files
that contain what I'm looking for right now.
And if I click on any of them,
we have the request and the response,
and we can actually use the same regex
to search for parameters within the request,
and the response.
So at this stage,
I would actually go through these one by one.
So, you can already see, the generate credentials.
You can already see, the admin.
If we go to the profile, for example,
and search for the same pattern,
you can see, we're only matching the pattern
that we're looking for.
So, we're only matching a question mark,
followed by something, followed by an equal,
but these are at Google API.
So, it's outside of the scope.
It's not actually part of my target.
So, we're gonna be skipping these,
because they're outside of our scope.
And then eventually, you're actually gonna get
to some interesting stuff.
So, we actually tested this before the ID parameter,
and it was actually vulnerable to an eye door.
And then if you continue,
you might even find more interesting stuff,
but there isn't in this file.
So at this stage, I would actually go ahead,
and move on to the next file,
and analyze what's in it?
Now, I'm not gonna waste your time,
going through all of them,
because we've actually tested a lot of these.
So for example, you have the act login in here,
that we discovered an XSS in it as well,
but I'm gonna go straight to the menu,
because this page actually contains a parameter
that we have not tested.
So again, looking in the response,
we're gonna click on next,
and we have some stuff that is outside of our scope.
So, I'm gonna just skip them.
Right here, we're matching a pattern where we have a class.
So, this is just normal HTML markup.
It's nothing to test for us.
So, we're looking for parameters and values.
So, that's why we're skipping these, and perfect.
Right here, we have an actual parameter
that we have not tested yet.
So, this parameter is in the book dot php page,
and the parameter name is promoCode.
And currently as default, it is taking a value of UKONLY,
but obviously, as usual,
we're gonna be manipulating this,
to test for all of the bugs and vulnerabilities
that we've learned so far.
And we're gonna do that in the next lecture,
because the point of this lecture was actually,
to introduce you to regular expressions
and how to use them within burp
to discover hidden parameters,
and hidden endpoints.