When control-c cancels a scan, it should terminate reports and call report_end. When testing on the main branch a terminated scan writes the json file as it should. However, when terminating with this branch it does not. I'm not immediately spotting why.

I assume that you mean report_close, not report_end?

If so, this is strange because I am not experiencing anything like this on any branch. When I press Control + C, the json is saved normally, as usual.

If I am not mistaken safe_quit is responsible for graceful shutdown and it invokes report_host_end, report_summary and report_close in this order. I did not change logic within json_close (equivalent to report_close) - only string content (maybe lack of newline is a problem, but I don't see why?).

If you can pinpoint a commit that breaks graceful shutdown I could look into it

The mistake was on my end and an error I didn't notice.

@sullo @HuntClauss is this another breaking change to the JSON report format?
If yes, is there another release planned for this?
I would like to get the Nikto parser in DefectDojo updated for the JSON format from 2.5.0 and wonder if this will break it again.

@sullo @HuntClauss is this another breaking change to the JSON report format?

I don't believe so. This was fixing some bugs that made it invalid JSON and cleaning up how it operates to ensure the JSON is closed correctly. For example, the message may have had stray unquoted characters.

@sullo Earlier I run two scans, one using the master branch and one using the 2.5.0 release tag.

The report from 2.5.0 turned out to be invalid as expected (thats what this fix is for?):

{"host":"REDACTED","ip":"REDACTED","port":"443","banner":"Apache","vulnerabilities":[{"id": "999103","references": "https://www.netsparker.com/web-vulnerability-scanner/vulnerabilities/missing-content-type-header/","method":"GET","url":"","msg":""/: The X-Content-Type-Options header is not set. This could allow the user agent to render the content of the site in a different fashion to the MIME type.""},{"id": "999992","references": "https://en.wikipedia.org/wiki/Wildcard_certificate","method":"GET","url":"","msg":""Server is using a wildcard certificate: REDACTED.""}]}

The report form the master branch was valid:

[
  {
    "host": "REDACTED",
    "ip": "REDACTED",
    "port": "443",
    "banner": "",
    "vulnerabilities": [
      {
        "id": "999992",
        "references": "https://en.wikipedia.org/wiki/Wildcard_certificate",
        "method": "GET",
        "url": "/",
        "msg": "Server is using a wildcard certificate: REDACTED."
      }
    ]
  }
]

However, notice how the master branch one is an array at the top level, where the other one is a single object?

@HuntClauss I think that was to fix the multiple host issue--can you confirm?

I'm thinking more the report should just be done using a perl json library to ensure consistency and accuracy... but that will be a release if done.

@sullo Well, primarily my goal was to fix json report for one host only, because I didn't even know that nikto can scan more than one. But during implementation of this and couple other fixes I found out that it support multiple hosts so I changed json format accordingly.

@HuntClauss Thank you for your reply and the fixes :)
@sullo Currently this leaves me in an unclear state:

• 2.5.0 branch: doesn't work because it misses this fix Fix nikto json plugin #807
• 2.5.0 tag: doesn't work because it misses the fixes from this PR
• master works great but is not supported in my dependencies (DefectDojo and SecureCodebox) and I would like to get them to update their parsers

Could we do a new release 2.5.1 which replaces 2.5.0 completely?
Otherwise people who want to change their parser, might just look at the 2.5.0 release and implement changes for a broken format.
They then will be hit with another necessary(!) but breaking change in the next release due to the encoding changes and the multi-host support.

@moxli thanks for the suggestions and I understand your pain. Give me a day to evaluate replacing it all with proper usage of a JSON module--to your point I'd rather do this one time and do a dot release than do it more than once.

@moxli one thing to clarify is that master, in my world, is always the working most current code and the 2.5.0 tag/branch are basically snapshots. I should probably archive the 2.5.0 branch as it's now obsolete.

I'm sure this is the non-recommended approach by most people 🤣

@sullo thank you very much for your reply :)

@sullo Did you come to a conclusion about a possible rewrite of the json reporting yet?:)

I'm sure this is the non-recommended approach by most people 🤣

Yep, that's absolutely correct 😩