Question

I have a crawler that for HTTPs switches over to Wininet.

This usually works well, but for a website I get error
ERROR_INTERNET_CLIENT_AUTH_CERT_NEEDED

Example code:

vErrorNone := HttpSendRequest(HttpOpen_Request, nil, 0, nil, 0);
if vErrorNone = False then
  begin
    vErrorID := GetLastError;
    if (vErrorID = ERROR_INTERNET_CLIENT_AUTH_CERT_NEEDED) then
      begin
        // this error
      end
    ;
  end
;

I then tried an experiment with:

      TmpFakePointer := nil;
      vErrorNone :=
        InternetErrorDlg(
          GetDesktopWindow()
          ,
          HttpOpen_Request
          ,
          ERROR_INTERNET_CLIENT_AUTH_CERT_NEEDED
          ,
          0
          or FLAGS_ERROR_UI_FILTER_FOR_ERRORS
          or FLAGS_ERROR_UI_FLAGS_GENERATE_DATA
          or FLAGS_ERROR_UI_FLAGS_CHANGE_OPTIONS
          ,
          TmpFakePointer
        )
        = ERROR_SUCCESS
      ;
      if vErrorNone then
        begin
          vErrorNone := HttpSendRequest(HttpOpen_Request, nil, 0, nil, 0);
        end
      ;

However, two things are odd here:

• 1) No dialog shows
• 2) It works
• 2.1) No error on second HttpSendRequest
• 2.2) Valid data returned

It find above combination very odd and contrary to documentation. Since user has not selected any certificate, why does this work? If Wininet falls back to anonymous client authentication certificate when the dialog can not be shown (I am assuming I have given it incorrect handle to show dialog? Although it errors if I explicitly give it a wrong handle) is there any way to pick that directly without the call to InternetErrorDlg?

Answer 1

Well

There are two things you can do:

1: Show the certificate error to the user and let him decide or not to continue.
2: Ignore any certificate errors you get.

I am gonna show you how to do the second option:

When you get the ERROR_INTERNET_CLIENT_AUTH_CERT_NEEDED or any other certificate error you need to call InternetSetOptions to tell wininet that it should ignore the error and continue. After that, you have to re-send the request.

function SetToIgnoreCerticateErrors(var aErrorMsg: string): Boolean;
var
  vDWFlags: DWord;
  vDWFlagsLen: DWord;
begin
  Result := False;
  try
    vDWFlagsLen := SizeOf(vDWFlags);
    if not InternetQueryOptionA(oRequestHandle, INTERNET_OPTION_SECURITY_FLAGS, @vDWFlags, vDWFlagsLen) then begin
      aErrorMsg := 'Internal error in SetToIgnoreCerticateErrors when trying to get wininet flags.' + GetWininetError;
      Exit;
    end;
    vDWFlags := vDWFlags or SECURITY_FLAG_IGNORE_UNKNOWN_CA or SECURITY_FLAG_IGNORE_CERT_DATE_INVALID or SECURITY_FLAG_IGNORE_CERT_CN_INVALID or SECURITY_FLAG_IGNORE_REVOCATION;
    if not InternetSetOptionA(oRequestHandle, INTERNET_OPTION_SECURITY_FLAGS, @vDWFlags, vDWFlagsLen) then begin
      aErrorMsg := 'Internal error in SetToIgnoreCerticateErrors when trying to set wininet INTERNET_OPTION_SECURITY_FLAGS flag .' + GetWininetError;
      Exit;
    end;
    Result := True;
  except
    on E: Exception do begin
      aErrorMsg := 'Unknown error in SetToIgnoreCerticateErrors.' + E.Message;
    end;
  end;
end;


vErrorNone := HttpSendRequest(HttpOpen_Request, nil, 0, nil, 0);
if vErrorNone = False then
  begin
    vErrorID := GetLastError;
    if (vErrorID = ERROR_INTERNET_CLIENT_AUTH_CERT_NEEDED) then
    begin
      //call SetToIgnoreCerticateErrors
      //re-send the request 
    end
  end
end;

I have extracted the SetToIgnoreCerticateErrors from my wininet API and it may not compile exactly how it is.

Those are the steps:

1 – Get the error
2 – Check if the error is an certificate error
3 – If it is a certificate error, them call InternetSetOption as I did.
4 – Re-send the request.

I don’t know how to implement the first option “Show the certificate error to the user and let him decide or not to continue.” because I never had to do it.

Also, check this out: How To Handle Invalid Certificate Authority Error with WinInet

I hope it helps you.