Accuracy measures how close is a value to the true/actual value. Who cares?
Accuracy is a professed desire of most users of valuation services. However, today’s conflicted regulations, legacy practices, and obsolete standards — do not have accuracy as a goal!
The overarching goal of appraisal today is not “be accurate” — it is “be credible,” defined in USPAP as “worthy of belief.” In fact, the word ‘accuracy’ is nowhere to be found in the ethical standards, nor in real property standards. There are only three work-scope requirements to be found: 1) be believable; 2) meet user expectations; and 3) do what peers would do.
Fortunately, the appraisal-development standard is stable. There has been only one word changed in over five years — this, in a world of rapidly evolving data analysis technology. Be credible, don’t change.
However, in Standard 3, appraisal reviewers are required to develop an opinion of accuracy as well as of “reasonableness.” Can an accurate result be unreasonable? Apparently yes. The review standard assumes that the reviewer has access to the actual, true value – against which the appraiser’s believable value can be compared. So, if the credible value does not match the reviewer’s true opinion of actual value, the original appraiser is clearly wrong, not credible, and thus in violation of USPAP!
Credibility is subjective, and cannot be measured! One opinion judging another opinion!
Accuracy can only be measured in two ways:
- Compare the ‘credible’ value against the actual, true, known value.
- Assess the process for its objectivity – the data used, and the algorithm (logic and math).
Since we never know a true “most probable” market price, we must focus on establishing a true process.
What will a “true process” for appraisal accuracy look like?
- The problem — assumptions and goal of analysis must be clear, and answer the question asked.
- The data — must be the right data. All the relevant available data, and no ‘garbage’ data.
- The algorithm – must match the optimal economic/behavioral model.
For real property valuation:
- The right problem may be to identify the market price, insurable value, quick-sale value, perhaps fundamental value, loan-risk value, or reparative value.
- The right data ideal is the Competitive Market Segment (CMS©), if there is “sufficient” data. This can be supplemented by indirectly competitive sales. In our Evidence Based Valuation (EBV)© curriculum, we apply the five dimensions of similarity, toward the several similarity tools available in the data science approach to valuation.
- The right algorithm reflects the right model. We find that although today’s algorithms are powerful, fast, and much easier to understand – expert judgment is totally required.
Today, the asset analyst must understand and apply the right data, the right model, and the right algorithm. And communicate the analysis and results to the client, including a risk/reliability score.
This is accomplished through competency in data concepts, using visuals (graphs), and the interplay of refined expert judgment through critical thinking.
Form-filler appraisers will soon be passed over. Competent reliability/valuation analysts are needed. Skilled analysts have the foundation of skills and knowledge to best serve the public trust.
The CAA is currently evolving the EBV technical standards book of practice, and designations reflecting the above competencies in asset analytics and delivery. (Community of Asset Analysts). Join us!