Advance HTTP - Nuclei Documentation

We’ve enriched nuclei to allow advanced scanning of web servers. Users can now use multiple options to tune HTTP request workflows.

Pipelining

HTTP Pipelining support has been added which allows multiple HTTP requests to be sent on the same connection inspired from http-desync-attacks-request-smuggling-reborn. Before running HTTP pipelining based templates, make sure the running target supports HTTP Pipeline connection, otherwise nuclei engine fallbacks to standard HTTP request engine. If you want to confirm the given domain or list of subdomains supports HTTP Pipelining, httpx has a flag -pipeline to do so. An example configuring showing pipelining attributes of nuclei.

    unsafe: true
    pipeline: true
    pipeline-concurrent-connections: 40
    pipeline-requests-per-connection: 25000

An example template demonstrating pipelining capabilities of nuclei has been provided below-

id: pipeline-testing
info:
  name: pipeline testing
  author: pdteam
  severity: info

http:
  - raw:
      - |+
        GET /{{path}} HTTP/1.1
        Host: {{Hostname}}
        Referer: {{BaseURL}}

    attack: batteringram
    payloads:
      path: path_wordlist.txt

    unsafe: true
    pipeline: true
    pipeline-concurrent-connections: 40
    pipeline-requests-per-connection: 25000

    matchers:
      - type: status
        part: header
        status:
          - 200

Connection pooling

While the earlier versions of nuclei did not do connection pooling, users can now configure templates to either use HTTP connection pooling or not. This allows for faster scanning based on requirement. To enable connection pooling in the template, threads attribute can be defined with respective number of threads you wanted to use in the payloads sections. Connection: Close header can not be used in HTTP connection pooling template, otherwise engine will fail and fallback to standard HTTP requests with pooling. An example template using HTTP connection pooling-

id: fuzzing-example
info:
  name: Connection pooling example
  author: pdteam
  severity: info

http:

  - raw:
      - |
        GET /protected HTTP/1.1
        Host: {{Hostname}}
        Authorization: Basic {{base64('admin:§password§')}}

    attack: batteringram
    payloads:
      password: password.txt
    threads: 40

    matchers-condition: and
    matchers:
      - type: status
        status:
          - 200

      - type: word
        words:
          - "Unique string"
        part: body

Smuggling

HTTP Smuggling is a class of Web-Attacks recently made popular by Portswigger’s Research into the topic. For an in-depth overview, please visit the article linked above. In the open source space, detecting http smuggling is difficult particularly due to the requests for detection being malformed by nature. Nuclei is able to reliably detect HTTP Smuggling vulnerabilities utilising the rawhttp engine. The most basic example of an HTTP Smuggling vulnerability is CL.TE Smuggling. An example template to detect a CE.TL HTTP Smuggling vulnerability is provided below using the unsafe: true attribute for rawhttp based requests.

id: CL-TE-http-smuggling

info:
  name: HTTP request smuggling, basic CL.TE vulnerability
  author: pdteam
  severity: info
  reference: https://portswigger.net/web-security/request-smuggling/lab-basic-cl-te

http:
  - raw:
    - |+
      POST / HTTP/1.1
      Host: {{Hostname}}
      Connection: keep-alive
      Content-Type: application/x-www-form-urlencoded
      Content-Length: 6
      Transfer-Encoding: chunked
      
      0
      
      G      
    - |+
      POST / HTTP/1.1
      Host: {{Hostname}}
      Connection: keep-alive
      Content-Type: application/x-www-form-urlencoded
      Content-Length: 6
      Transfer-Encoding: chunked
      
      0
      
      G
            
    unsafe: true
    matchers:
      - type: word
        words:
          - 'Unrecognized method GPOST'

More examples are available in template-example section for smuggling templates.

Race conditions

Race Conditions are another class of bugs not easily automated via traditional tooling. Burp Suite introduced a Gate mechanism to Turbo Intruder where all the bytes for all the requests are sent expect the last one at once which is only sent together for all requests synchronizing the send event. We have implemented Gate mechanism in nuclei engine and allow them run via templates which makes the testing for this specific bug class simple and portable. To enable race condition check within template, race attribute can be set to true and race_count defines the number of simultaneous request you want to initiate. Below is an example template where the same request is repeated for 10 times using the gate logic.

id: race-condition-testing

info:
  name: Race condition testing
  author: pdteam
  severity: info

http:
  - raw:
      - |
        POST /coupons HTTP/1.1
        Host: {{Hostname}}

        promo_code=20OFF        

    race: true
    race_count: 10

    matchers:
      - type: status
        part: header
        status:
          - 200

You can simply replace the POST request with any suspected vulnerable request and change the race_count as per your need, and it’s ready to run.

nuclei -t race.yaml -target https://api.target.com

Multi request race condition testing For the scenario when multiple requests needs to be sent in order to exploit the race condition, we can make use of threads.

    threads: 5
    race: true

threads is a total number of request you wanted make with the template to perform race condition testing. Below is an example template where multiple (5) unique request will be sent at the same time using the gate logic.

id: multi-request-race

info:
  name: Race condition testing with multiple requests
  author: pd-team
  severity: info

http:
  - raw:  
      - |
        POST / HTTP/1.1
        Pragma: no-cache
        Host: {{Hostname}}
        Cache-Control: no-cache, no-transform
        User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0

        id=1
        
      - |
        POST / HTTP/1.1
        Pragma: no-cache
        Host: {{Hostname}}
        Cache-Control: no-cache, no-transform
        User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0

        id=2

      - |
        POST / HTTP/1.1
        Pragma: no-cache
        Host: {{Hostname}}
        Cache-Control: no-cache, no-transform
        User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0

        id=3

      - |
        POST / HTTP/1.1
        Pragma: no-cache
        Host: {{Hostname}}
        Cache-Control: no-cache, no-transform
        User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0

        id=4

      - |
        POST / HTTP/1.1
        Pragma: no-cache
        Host: {{Hostname}}
        Cache-Control: no-cache, no-transform
        User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0

        id=5

    threads: 5
    race: true

Requests Annotation

Request inline annotations allow performing per request properties/behavior override. They are very similar to python/java class annotations and must be put on the request just before the RFC line. Currently, only the following overrides are supported:

@Host: which overrides the real target of the request (usually the host/ip provided as input). It supports syntax with ip/domain, port, and scheme, for example: domain.tld, domain.tld:port, http://domain.tld:port
@tls-sni: which overrides the SNI Name of the TLS request (usually the hostname provided as input). It supports any literals. The special value request.host uses the Host header and interactsh-url uses an interactsh generated URL.
@timeout: which overrides the timeout for the request to a custom duration. It supports durations formatted as string. If no duration is specified, the default Timeout flag value is used.

The following example shows the annotations within a request:

- |
  @Host: https://projectdiscovery.io:443
  POST / HTTP/1.1
  Pragma: no-cache
  Host: {{Hostname}}
  Cache-Control: no-cache, no-transform
  User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0

This is particularly useful, for example, in the case of templates with multiple requests, where one request after the initial one needs to be performed to a specific host (for example, to check an API validity):

http:
  - raw:
      # this request will be sent to {{Hostname}} to get the token
      - |
        GET /getkey HTTP/1.1
        Host: {{Hostname}}
        
      # This request will be sent instead to https://api.target.com:443 to verify the token validity
      - |
        @Host: https://api.target.com:443
        GET /api/key={{token}} HTTP/1.1
        Host: api.target.com:443

    extractors:
      - type: regex
        name: token
        part: body
        regex:
          # random extractor of strings between prefix and suffix
          - 'prefix(.*)suffix'

    matchers:
      - type: word
        part: body
        words:
          - valid token

Example of a custom timeout annotations -

- |
  @timeout: 25s
  POST /conf_mail.php HTTP/1.1
  Host: {{Hostname}}
  Content-Type: application/x-www-form-urlencoded
  
  mail_address=%3B{{cmd}}%3B&button=%83%81%81%5B%83%8B%91%97%90M

Example of sni annotation with interactsh-url -

- |
  @tls-sni: interactsh-url
  POST /conf_mail.php HTTP/1.1
  Host: {{Hostname}}
  Content-Type: application/x-www-form-urlencoded
  
  mail_address=%3B{{cmd}}%3B&button=%83%81%81%5B%83%8B%91%97%90M

Template Guide

​Pipelining

​Connection pooling

​Smuggling

​Race conditions

​Requests Annotation

Pipelining

Connection pooling

Smuggling

Race conditions

Requests Annotation