Collection of References on Why Password Policies Need to Change

Organisations like NIST and the UK National Cyber Security Centre (NCSC) are pushing password security policies that are much different from the past. Most notably, password expiry and character composition rules are being dropped, and replaced by other more user friendly recommendations.  Despite their efforts, many organisations are very slow in changing to the modern guidance, and instead remain with password policy practices that are characteristic of 2004 guidance. If you’d like to work towards change in your organisation, it helps to have a useful set of references to pass on to those who write the policies.

Is it worth trying to make the change? In my judgement, an important part of building a positive security culture is considering pain versus value in every decision you make. A lot of historical guidance for password security is high pain and little to no security value according to what we have learned, and sometimes causes more harm than good in ways that policy makers never anticipated. We also must remember that availability is one of the three pillars of security: when users get locked out of their account for reasons that can at least partially be attributed to password security policies, then we are not putting the best face of security forward.  This is especially true when there are better ways to do things:  Security policy writers need to keep their knowledge up to date.

This blog contains a information from various sources including research papers, reports/white papers, popular blogs, government websites, and news organisations. Dates of publications are included so readers can keep in mind that more recent publications are based upon more recent knowledge. While there are many existing blogs on password security, the main contributions here are:

1. Assembling a large number of modern sources together,
2. Providing compact summaries of why the sources are relevant to the purpose of making password policy change within an organisation,
3. Including research publications that show where new recommendations are derived from (thus going beyond “appeal to authority” arguments — the research is there for anybody who cares to check it themselves).

Overview of modern password security guidance

• (WSJ news article – requires subscription, 2017) The Man Who Wrote Those Password Rules Has a New Tip: N3v$r M1^d!. This article talks about how the author of the classic NIST document that proposed composition rules and similar guidance (from NIST Special Publication 800-63 Appendix A) now rejects those recommendations. Those recommendations seemed to be okay back then, but by what we have learned today, they no longer serve the original intent. New recommendations are written that balance security with usability needs.
• (Naked Security news article, 2016) NIST’s new password rules – what you need to know. This is a compact summary of the changes to NIST’s password security guidance. Because the article is short, it is a good way to open up the conversation with people who have limited time or attention span. It tells what policy makers need to change to be in compliance with NIST guidelines.
• (Troy Hunt blog, 2017) Passwords Evolved: Authentication Guidance for the Modern Era. This is quite a long, but very well written article about the changes to NIST’s password guidance and why the changes are being made. It also draws from the UK government’s guidance and Microsoft’s guidance to strengthen the argument. The section on “Listen to Your Governments (and Smart Tech Companies)” makes a great appeal to authority argument on why companies should take these recommendations seriously. Overall, it is a good read for those who have time and interest to read it.

The new recommendations — original sources

• (NIST publication, 2017) NIST Special Publication 800-63B. This is right from the horse’s mouth, but it’s not a document to start the conversation with because the document is large and uses terminology that will take time for many readers to interpret. It is nevertheless the proof you will need if people question whether NIST is really making such a recommendation. For example, the requirement not to expire have composition rules for passwords (“memorized secrets”) is given in section 10.2.1. That section also says these secrets should not be required to change periodically. The requirement to not allow hints to recover passwords is given in section 5.1.1.2. The requirement to not allow secret questions for account recovery is better found in NIST’s FAQ.
• (NIST web page FAQ, 2019) NIST Special Publication 800-63: Digital Identity Guidelines Frequently Asked Questions. This is an easier place to start than the NIST original publication, as it is more human readable. It gets right to the point on questions like why composition rules are no longer recommended, why password expiry is no longer recommended, and why secret questions for account recovery are no longer permitted.
• (NCSC publication, 2018) Password administration for system owners. These recommendations are similar in nature to the NIST recommendations. See sections on “Don’t enforce regular password expiry” and “Do not use complexity requirements”. But additionally worth pointing out is “Reduce your organisation’s reliance on passwords” (we have too many passwords already) and “Implement technical solutions”, which includes other ways guidance on helping protect user passwords while not locking users out of their accounts long-term.
• (NCSC infographic, 2018) NCSC Password Policy Advice for system owners. A simple graphic that shows the risks and how to help improve password security within an organisation. The right half in purple gives password policy recommendations (unfortunately, while most of the infographic is good, there is one serious blunder in it where it recommends SHA-256 for password hashing. SHA256 was not designed for this type operation. Those who know the cryptography understand this, those who don’t can get the understanding from an old Troy Hunt article).
• (NCSC blog, 2016) The problems with forcing regular password expiry.  A simple blog that explains why password expiry causes more harm than good. In short, users choose similar passwords to previous passwords and minimally meet the complexity requirement to try create a password they can memorise. There is also an increased tendency of forgotten passwords, that causes a productivity cost to an organisation.
• (Microsoft publication, 2016) Microsoft Password Guidance. This report is great, because it is easy to read and gets right to the point. On the first page it enumerates 7 recommendations to system administrators which include things not to do (do not enforce password expiry or composition rules) and what to do (8 character minimum password, ban common passwords, etc…). Further in the document it goes into more details and includes references to research reports that justify why the changes are being made. Overall, the clarity, simplicity, and completeness (especially research references) of this publication make it a top source for referencing to others. However, because it predates the NIST and NCSC changes, it lacks the references to NIST’s new guidelines. It also lacks details on how organisations can implement the risk based multi factor authentication that it recommends.

Research on why password policies need to change

• (Research paper, 2010) The Security of Modern Password Expiration: An Algorithmic Framework and Empirical Analysis. The main goal of password expiry is limiting the amount of time an attacker has access to an account in the event of password compromise. This paper questions whether that goal is met by analysing a dataset of 7700 accounts to determine whether knowing one password allowed recovering other passwords for that user. The fallacy around password expiry recommendations is that security administrators assume users choose each password randomly and independently from other passwords, but the reality is that the vast majority of users don’t . The paper shows even for websites where users have an incentive in protecting their accounts (for example, it holds payroll data), new user passwords tend to be strongly related to previous ones. The study found that could derive 17% of new passwords within 5 guesses and 41% of new passwords within seconds in an offline attack with little effort. The authors write:
  

  “Combined with the annoyance that expiration causes users, our evidence suggests it may be appropriate to do away with password expiration altogether, perhaps as a concession while requiring users to invest the effort to select a significantly stronger password than they would otherwise (e.g., a much longer passphrase).”

• (Research paper, 2010) Testing Metrics for Password Creation Policies by Attacking Large Sets of Revealed Passwords.  When NIST wrote their 2004 Special Publication 800-63 that had the recommendations for password policy, they included entropy estimates on how strong the passwords complying to the policy would be. This research does password cracking on sets of real user data to prove that those estimates are far too high. Part of the reason for this is that users tend to follow similar patterns when forced to comply with a password policy. For example, when required to use digits, users tend to either choose all digits or else put the digits at the end of the password (nearly 85%); When required to use an upper case letter, users tend to use either all upper case or put only the first letter as upper case (89%); and when required to use a special character users tend to put it at the end of the password (28.5%). An attacker can use known human behaviour to increase his chances of success for password cracking attacks. The paper notes that using a password blacklist of 50,000 passwords helps significantly, but not as much as NIST predicted. The authors conclude
  

  “Our findings were that … most common password creation policies remains vulnerable to online attack. This is due to a subset of the users picking easy to guess passwords that still comply with the password creation policy in place, for example “Password!1” ”

• (Research paper, 1999) Users are not the Enemy.  This paper surveys a large number of users to understand the problems they have with password security. While it is true that many users do not assume the responsibility they should for protecting their accounts, the study also finds that part of the problem is the difficulty users have with complying with password security policies. Users have a large number of passwords that need to change regularly and each having different password complexity requirements, which makes it much more likely that users will do things that they should not do just so they do not lose access to their accounts. The section on Security needs user-centered design notes that
  

  “Many of these [security] mechanisms create overheads for users, or require unworkable user behavior. It is therefore hardly surprising to find, that many users try to circumvent such mechanisms.”

  The paper concludes with a set of recommendations to help users with passwords. However, this paper is old (from 1999) so therefore the recommendations are also subject to the knowledge of the time.

• (Research paper, 2010) The True Cost of Unusable Password Policies: Password Use in the Wild.  The authors note that users are generally cogent in their understanding of security needs, but found compliance to passwords policies too difficult. To cope with security demands, users developed their own strategies, which end up introducing their own problems. As a consequence, the complexity of complying with security policies results in an adverse effect on the security posture of the organisations. For example, one user dealt with a policy that forced him to change his password in a way that it was not similar to any 12 of his previous passwords by just writing down the password so he would not forget it. In the section on Towards Holistic Password Policies, the authors note that just looking at the technical side and ignoring the user side, does not encourage security awareness — instead it introduces problems that antagonise users. Instead, password policies need to be design for the context in which users use the systems for, with an emphasis on eliminating the risks that they are likely to face in that context.
• (Research paper, 2007) Do Strong Web Passwords Accomplish Anything?. There are many ways that an attacker may go after a user, and password policies only address defence against brute forcing attacks. They do not address phishing, key logging, shoulder surfing, insecure password storage on local machines, and guessing based upon special knowledge about the user. Although strong passwords do help in some cases, there are other more user-friendly security controls that could be put in place of the complex password policy. These other controls make strong password policies less important. The paper argues
  

  “Since the cost is borne by the user, but the benefit is enjoyed by the bank user resistance to stronger passwords is predictable. We argue that there are better means of addressing brute-force bulk guessing attacks.“

  However this paper is from 2007. See next item which is related but more recent:

• (Blog, 2019) Your Pa$$word doesn’t matter.  This blog unfortunately has a misleading title that caused many readers to misjudge it before reading it. It is not telling users not to choose strong passwords, but instead is saying that password complexity policies have little benefit when considering the various ways that passwords are attacked. In spirit it is similar to the previous item, but this research is more modern. It tells system administrators and security policy writers:
  

  “Focusing on password rules, rather than things that can really help – like multi-factor authentication (MFA), or great threat detection – is just a distraction.”

• (Research paper, 2015) Quantifying the Security Advantage of Password Expiration Policies. The abstract gets right to the point:
  

  “Many security policies force users to change passwords within fixed intervals, with the apparent justification that this improves overall security. However, the implied security benefit has never been explicitly quantified. In this note, we quantify the security advantage of a password expiration policy, finding that the optimal benefit is relatively minor at best, and questionable in light of overall costs.”

  This paper does a mathematical analysis on the benefit of password expiry policies. Without considering the possibility of new passwords being related to old, this paper instead considers an attacker who keeps trying to guess the user password in an online attack even after the password might have changed without the attackers knowledge of that change happening. In essence, they show that the attacker’s chance of success is not largely different than if there was no password expiry policy in place. And this is even if the password is chosen randomly (most users don’t do this complexity, which makes attacks easier). The authors conclude by challenging those who favour such policies to explain why and in which specific circumstances a substantiating benefit is evident.

• (Research paper, 2009) It’s no secret : Measuring the security and reliability of authentication via ‘secret’ questions.  Historically, requiring users to provide answers to secret questions upon registration was a way to do account recovery in the event of forgotten password. This paper analyses these practices and finds that 20% of users forget their own answers to secret questions within 6 months, acquaintances of such people can guess answers to their secret questions 17% of the time, and 13% of the time an answer could be guessed by an attacker within 5 attempts by trying the most popular answers. The authors conclude that these questions are neither reliable nor do they meet security requirements. They propose a number of options to improve secret questions or have alternative backup authenticators, but ultimately this paper more than any other lead to the removal of secret questions for account recovery.

Other references

• (Microsoft blog, 2019) Security baseline (FINAL) for Windows 10 v1903 and Windows Server v1903.  Periodic password expiration will no longer be enabled in Windows 10 and Windows Server. The blog writes
  

  “Periodic password expiration is an ancient and obsolete mitigation of very low value, and we don’t believe it’s worthwhile for our baseline to enforce any specific value. By removing it from our baseline rather than recommending a particular value or no expiration, organizations can choose whatever best suits their perceived needs without contradicting our guidance. At the same time, we must reiterate that we strongly recommend additional protections even though they cannot be expressed in our baselines.”

• (FTC blog, 2016) Time to rethink mandatory password changes.  This is from a former Chief Technologist of the US Federal Trade Commission. The author explains why requiring users to change their passwords does more harm than good, and includes a long list of references like many included here to back up the argument. While there may be reasons for you to change your password (examples given), requiring regular changes in a password policy is not necessarily good practice:
  

  “Research suggests frequent mandatory expiration inconveniences and annoys users without as much security benefit as previously thought, and may even cause some users to behave less securely. Encouraging users to make the effort to create a strong password that they will be able to use for a long time may be a better approach for many organizations…“

Concluding remarks

The shortcomings of legacy password policies are well documented, however there are two distinct philosophies for moving forward.  On one hand, there remains the “fix the user” philosophy which pushes education and password managers as the main mechanisms for protecting user accounts.  The alternate approach is to design systems that are less reliant upon passwords as the sole determinant for authentication, which is the approach that Google and Microsoft seem to be taking (related: see Protecting User Accounts When Usability Matters).  I honestly believe that both philosophies have a place going forward.  The problem today is that there is way too much emphasis on the former and not enough effort being put into the latter.  We can’t always count on people to do the right thing, but there are often things we can do to protect their backs when they are negligent.  Putting security complexity burden on the user should be the last fallback option, not the default option.