Experiencing Occasional 'Basic Authentication Failed' errors under SSO

Options

We recently switched majority of our workspaces over from Basic Authentication to SSO for security. To populate our models with data that is rapidly changing we push data through our internal data store through an ETL (Workiva formerly known as OneCloud) hourly sometimes more often than that. There is a dedicated service account that is used and labeled as a Workspace Admin and SSO Exception user allowing it to sign in with the traditional Basic Authentication which was done across every workspace. When the switch was made to SSO each workspace/model connection was validated for each workspace with no issues end to end initially.

We are now seeing occasional errors while syncing our data with the connector returning a 'Basic Authentication Failed' error < 10% of the time during the BD. None of the configurations changed other than the using SSO. The Authentication & API Basic URI are both still the defaults at (https://api.anaplan.com) & (https://auth.anaplan.com)

I was curious if this was a known error or is there another action that need to be completed to eliminate this error from happening sporadically.

Any help would be very beneficial.

Answers

  • QuinE
    Options

    Hi @ccrowe1 - I have seen this issue before, and typically it has to do with Workiva's secrets manager giving the wrong credentials. From the sounds of it, you have everything set up correctly. I would contact Workiva support and see if they perhaps had an issue with their secrets during the time you experienced this issue.

  • ccrowe1
    Options

    Hi @QuinE, thank you I will explore this with Workiva.

  • ccrowe1
    Options

    Hi @QuinE,

    The response I got back from Workiva support was that 'They said we don't have a "Secrets Manager" that they have ever heard of.'

    Is there another term they may use for this now? I know they have made some changes since the platform was OneCloud.

  • QuinE
    Options

    Hi @ccrowe1 - On the Workiva platform, they will have a way to save your login credentials that you save with the WChains connection manager. The system that saves these credentials might have had a service disruption coinciding with your issue. This issue might be listed here: https://status.workiva.com/ . Unfortunately, this is all we can offer. You can also look at https://status.anaplan.com/ to see if there were any API issues when your integration in Workiva was being executed.

  • Hi @ccrowe1

    I have an open ticket with Anaplan support about a similar issue running our exports via Anaplan Connect.

    We have been getting these errors periodically but following one of the Anaplan platform updates the frequency of these errors has significantly increased.

    Anaplan support did imply that they were waiting on a fix from one of their vendors. Since then they have taken a copy of our model for further investigation.

    We are running the same script repeatedly throughout office hours. Sometimes it fails and sometimes it works. We are testing the latest version of APC and we get the same error in both UAT and Production at the same time.

    Interesting that we both had an issue at the same time.

    Using Class-Path: C:\Anaplan\anaplan-connect\anaplan-connect-1.4.4-jar-with-dependencies.jar
    2024-03-20 09:00:59 DEBUG [c.a.client.Program :1726 ] 12496 |-- ======================================================================
    2024-03-20 09:00:59 DEBUG [c.a.client.Program :1727 ] 12496 |-- Anaplan Connect 1.4.4
    2024-03-20 09:00:59 DEBUG [c.a.client.Program :1728 ] 12496 |-- Java HotSpot(TM) Client VM (Oracle Corporation)/ (25.391-b13)/
    2024-03-20 09:00:59 DEBUG [c.a.client.Program :1730 ] 12496 |-- (Windows Server 2016x86)/10.0
    2024-03-20 09:00:59 DEBUG [c.a.client.Program :1731 ] 12496 |-- ======================================================================
    2024-03-20 09:00:59 INFO [c.a.client.Service :83 ] 12496 |-- Initializing Service...
    2024-03-20 09:00:59 INFO [a.BasicAuthenticator:26 ] 12496 |-- Authenticating via Basic...
    2024-03-20 09:01:12 ERROR [c.a.client.Program :650 ] 12496 |-- Anaplan API: Basic Authentication failed! (Feign: status 500 reading AnaplanAuthenticationAPI#authenticateBasic(String); content:
    {
    "status" : "FAILURE_GENERIC",
    "statusMessage" : "Server failure"
    })

  • MikeMonRe
    Options

    Hi @ccrowe1

    I just wanted to share some of my findings. Before the weekend, I amended the schedule of our UAT process. Moving it away from 00 past the hour has eliminated all our failures. We have had a secondary process running in UAT at 30 minutes past the hour. This has never failed.

    Additionally, our scripts never fail at weekends they always succeed.

    I have sent these observations to Anaplan support and their investigations are still ongoing.

    Hopefully this will help your situation.

  • ccrowe1
    Options

    Hi @MikeMonRe,

    Thank you for sharing your findings!

    We built in some retry logic to execute some key steps again after a short pause that had been sporadically returning the authentication errors.

    This too had resolved the complete failures, but also does not remedy the root of the issue since the errors are still occurring on the first/second attempts.

    If this is an option for you, I would recommend looking into this as a temporary solution.

  • MikeMonRe
    Options

    @ccrowe1

    Thanks for the suggestion. I will look into the retry option. In the meantime I have a duplicate schedule running at a different time alongside the zero past the hour. This will be monitored so that I can see when Anaplan resolve the issue.

  • Glad to hear we are not alone in these struggles! We also have an open ticket with Anaplan support as we have been experience a sharp increase in integration failures beginning in March 2024 which co-incidentally aligns with the release of the new "log-on experience".

    The majority of our jobs start at the top of the hour (which seems to be common place) and the level 1 support team has indicated that peak traffic demands are contributing to the integration failures. They noted that is an enhancement targeted for mid-June to ensure that all integrations runs successfully regardless of when they are scheduled. No details provided on specifics of the enhancement but might be the same vendor fix you alluded to.

    The suggested interim fix is to update the scheduling away from top of the hour to avoid the peak traffic. This aligns with observations from @ccrowe1 @MikeMonRe though I will be very interested to learn more about root cause (what broke and why).