Natural Language Understanding (NLU) in Fraud Risk Management – a case study
This is a continuation of my previous blog, “Natural Language Understanding – Application Notes with Context Discriminant”.
Natural Language Understanding (NLU) is a subtopic of Natural Language Processing (NLP). Successful implementations of NLU are difficult because of limitations in prevailing technology. SiteFocus solved these limitations with a new approach to NLU. This approach has been successfully implemented into a commercial solution called Communications in Focus (CIF).
CIF is a cloud-based Artificial Intelligence solution that uses NLU to solve enterprise applications that center around natural language. It was commercially launched during the O’Reilly AI Conference in San Francisco (Sept. 18-20). SiteFocus was selected as one of 9 companies to present in the Startup Showcase during The AI Conference.
The purpose of this blog:
In this blog, we discuss a CIF analysis of a “fraud risk management” document. This use-case demonstrates the power of CIF and NLU to solve real-world problems in Fraud Risk Management.
We started our search for a test document with a report from KPMG, titled “Fraud Risk Management – Developing a strategy for prevention, detection, and response”. This report led us to a referenced document from the SEC.gov website – Accounting and Auditing Enforcement Release No. 1470 / October 23, 2001.
Below is the case background for this document, quoted from a SEC.gov website.
“Gisela de Leon-Meredith, age 37, resides in Pembroke Pines, Florida and acted as controller of Chestnut Hill Farms (“CHF”), a Miami-based division of Seaboard Corporation, a Delaware corporation headquartered in Shawnee Mission, Kansas. Seaboard’s shares are registered with the Commission under Section 12(b) of the Exchange Act and trade on the American Stock Exchange under the symbol SEB. Seaboard is a diversified international company primarily engaged in pork production and processing, transportation, energy generation, commodity brokering and produce farming. CHF is principally engaged in agricultural and shrimp production and marketing.
Meredith Manipulates CHF’s Assets and Expenses
A. One of the largest assets on CHF’s balance sheet is deferred farming costs, which are costs CHF has incurred in planting and growing produce and shrimp that it has not yet harvested. CHF capitalizes these costs as a deferred cost asset until harvesting begins, at which point CHF begins reducing the deferred cost asset by amortizing it on the basis of production results, hence expensing these costs to match them with revenues from the harvested crops.
B. From 1995 through the first quarter of 2000, Meredith booked improper entries in CHF’s books and records that overstated the deferred farming cost asset and understated farming expenses on CHF’s financial statements. Meredith knew by December 1998 that the entries were improper and that they had caused CHF’s deferred cost asset to be substantially overstated and CHF’s farming expenses to be understated, but, rather than disclose these errors to her supervisors or to Seaboard personnel, Meredith deliberately undertook to conceal the errors through other improper entries and adjustments.
C. Meredith’s deceptive efforts intensified in late 1999 as Seaboard personnel began inquiring about unusual entries in CHF’s monthly financial reports. Finally, in July 2000, after Seaboard’s internal auditing department questioned her about apparent discrepancies (now totaling over $7 million) in the deferred cost asset, Meredith confessed her wrongdoing to her immediate supervisor at CHF, who passed it along to Seaboard.”
From this case, we identified and submitted the content of the referenced SEC document – Accounting and Auditing Enforcement Release No. 1470 / October 23, 2001 – in verbatim without any edit into CIF for analysis.
The result of the analysis is provided by CIF within seconds.
We expect CIF to present an understanding of the content in the form of a context graph, namely, Meta-Vision, together with detailed references to context (excerpts from original text input) depicted by relevant subjects (Machine Generated Hash Tags or ‘MGHT’).
Below we include the (1) original input text and (2) screen captures from the CIF platform’s Meta-Vision analytics for readers to appreciate the capability of CIF’s automated NLU and insights that a “one click” analytics provides.
The value proposition of our approach with CIF is to use a simple visual metaphor (Meta-Vision) that enables anyone within an enterprise or government organization to take advantage of NLU in analyzing natural language communication – without dependency of data scientists or software engineers.
II. Input – Original Text
The following is the original text from the SEC document that was input into CIF for analysis:
“Today, we commence and settle a cease-and-desist proceeding against Gisela de Leon-Meredith, former controller of a public company’s subsidiary.1 Our order finds that Meredith caused the parent company’s books and records to be inaccurate and its periodic reports misstated, and then covered up those facts.
We are not taking action against the parent company, given the nature of the conduct and the company’s responses. Within a week of learning about the apparent misconduct, the company’s internal auditors had conducted a preliminary review and had advised company management who, in turn, advised the Board’s audit committee, that Meredith had caused the company’s books and records to be inaccurate and its financial reports to be misstated. The full Board was advised and authorized the company to hire an outside law firm to conduct a thorough inquiry. Four days later, Meredith was dismissed, as were two other employees who, in the company’s view, had inadequately supervised Meredith; a day later, the company disclosed publicly and to us that its financial statements would be restated. The price of the company’s shares did not decline after the announcement or after the restatement was published. The company pledged and gave complete cooperation to our staff. It provided the staff with all information relevant to the underlying violations. Among other things, the company produced the details of its internal investigation, including notes and transcripts of interviews of Meredith and others; and it did not invoke the attorney-client privilege, work product protection or other privileges or protections with respect to any facts uncovered in the investigation.
The company also strengthened its financial reporting processes to address Meredith’s conduct — developing a detailed closing process for the subsidiary’s accounting personnel, consolidating subsidiary accounting functions under a parent company CPA, hiring three new CPAs for the accounting department responsible for preparing the subsidiary’s financial statements, redesigning the subsidiary’s minimum annual audit requirements, and requiring the parent company’s controller to interview and approve all senior accounting personnel in its subsidiaries’ reporting processes.
Our willingness to credit such behavior in deciding whether and how to take enforcement action benefits investors as well as our enforcement program. When businesses seek out, self-report and rectify illegal conduct, and otherwise cooperate with Commission staff, large expenditures of government and shareholder resources can be avoided and investors can benefit more promptly.2 In setting forth the criteria listed below, we think a few caveats are in order:
First, the paramount issue in every enforcement judgment is, and must be, what best protects investors. There is no single, or constant, answer to that question. Self-policing, self-reporting, remediation and cooperation with law enforcement authorities, among other things, are unquestionably important in promoting investors’ best interests. But, so too are vigorous enforcement and the imposition of appropriate sanctions where the law has been violated. Indeed, there may be circumstances where conduct is so egregious, and harm so great, that no amount of cooperation or other mitigating conduct can justify a decision not to bring any enforcement action at all. In the end, no set of criteria can, or should, be strictly applied in every situation to which they may be applicable.
Second, we are not adopting any rule or making any commitment or promise about any specific case; nor are we in any way limiting our broad discretion to evaluate every case individually, on its own particular facts and circumstances. Conversely, we are not conferring any “rights” on any person or entity. We seek only to convey an understanding of the factors that may influence our decisions.
Third, we do not limit ourselves to the criteria we discuss below. By definition, enforcement judgments are just that — judgments. Our failure to mention a specific criterion in one context does not preclude us from relying on that criterion in another. Further, the fact that a company has satisfied all the criteria we list below will not foreclose us from bringing enforcement proceedings that we believe are necessary or appropriate, for the benefit of investors.
In brief form, we set forth below some of the criteria we will consider in determining whether, and how much, to credit self-policing, self-reporting, remediation and cooperation — from the extraordinary step of taking no enforcement action to bringing reduced charges, seeking lighter sanctions, or including mitigating language in documents we use to announce and resolve enforcement actions.
1. What is the nature of the misconduct involved? Did it result from inadvertence, honest mistake, simple negligence, reckless or deliberate indifference to indicia of wrongful conduct, willful misconduct or unadorned venality? Were the company’s auditors misled?
2. How did the misconduct arise? Is it the result of pressure placed on employees to achieve specific results, or a tone of lawlessness set by those in control of the company? What compliance procedures were in place to prevent the misconduct now uncovered? Why did those procedures fail to stop or inhibit the wrongful conduct?
3. Where in the organization did the misconduct occur? How high up in the chain of command was knowledge of, or participation in, the misconduct? Did senior personnel participate in, or turn a blind eye toward, obvious indicia of misconduct? How systemic was the behavior? Is it symptomatic of the way the entity does business, or was it isolated?
4. How long did the misconduct last? Was it a one-quarter, or one-time, event, or did it last several years? In the case of a public company, did the misconduct occur before the company went public? Did it facilitate the company’s ability to go public?
5. How much harm has the misconduct inflicted upon investors and other corporate constituencies? Did the share price of the company’s stock drop significantly upon its discovery and disclosure?
6. How was the misconduct detected and who uncovered it?
7. How long after discovery of the misconduct did it take to implement an effective response?
8. What steps did the company take upon learning of the misconduct? Did the company immediately stop the misconduct? Are persons responsible for any misconduct still with the company? If so, are they still in the same positions? Did the company promptly, completely and effectively disclose the existence of the misconduct to the public, to regulators and to self-regulators? Did the company cooperate completely with appropriate regulatory and law enforcement bodies? Did the company identify what additional related misconduct is likely to have occurred? Did the company take steps to identify the extent of damage to investors and other corporate constituencies? Did the company appropriately recompense those adversely affected by the conduct?
9. What processes did the company follow to resolve many of these issues and ferret out necessary information? Were the Audit Committee and the Board of Directors fully informed? If so, when?
10. Did the company commit to learn the truth, fully and expeditiously? Did it do a thorough review of the nature, extent, origins and consequences of the conduct and related behavior? Did management, the Board or committees consisting solely of outside directors oversee the review? Did company employees or outside persons perform the review? If outside persons, had they done other work for the company? Where the review was conducted by outside counsel, had management previously engaged such counsel? Were scope limitations placed on the review? If so, what were they?
11. Did the company promptly make available to our staff the results of its review and provide sufficient documentation reflecting its response to the situation? Did the company identify possible violative conduct and evidence with sufficient precision to facilitate prompt enforcement actions against those who violated the law? Did the company produce a thorough and probing written report detailing the findings of its review? Did the company voluntarily disclose information our staff did not directly request and otherwise might not have uncovered? Did the company ask its employees to cooperate with our staff and make all reasonable efforts to secure such cooperation?3
12. What assurances are there that the conduct is unlikely to recur? Did the company adopt and ensure enforcement of new and more effective internal controls and procedures designed to prevent a recurrence of the misconduct? Did the company provide our staff with sufficient information for it to evaluate the company’s measures to correct the situation and ensure that the conduct does not recur?
13. Is the company the same company in which the misconduct occurred, or has it changed through a merger or bankruptcy reorganization?
We hope that this Report of Investigation and Commission Statement will further encourage self-policing efforts and will promote more self-reporting, remediation and cooperation with the Commission staff. We welcome the constructive input of all interested persons. We urge those who have contributions to make to direct them to our Division of Enforcement. The public can be confident that all such communications will be fairly evaluated not only by our staff, but also by us. We continue to reassess our enforcement approaches with the aim of maximizing the benefits of our program to investors and the marketplace.
By the Commission (Chairman Pitt, Commissioner Hunt, Commissioner Unger).
1 In the Matter of Gisela de Leon-Meredith, Exchange Act Release No. 44970 (October 23, 2001).
2 We note that the federal securities laws and other legal requirements and guidance also promote and even require a certain measure of self-policing, self-reporting and remediation. See, e.g., Section 10A of the Securities Exchange Act of 1934, 15 U.S.C. § 78j-1 (requiring issuers and auditors to report certain illegal conduct to the Commission); In the Matter of W.R. Grace & Co., Exchange Act Release No. 39157 (Sept. 30, 1997) (emphasizing the affirmative responsibilities of corporate officers and directors to ensure that shareholders receive accurate and complete disclosure of information required by the proxy solicitation and periodic reporting provisions of the federal securities laws); In the Matter of Cooper Companies, Inc., Exchange Act Release No. 35082 (Dec. 12, 1994) (emphasizing responsibility of corporate directors in safeguarding the integrity of a company’s public statements and the interests of investors when evidence of fraudulent conduct by corporate management comes to their attention); In the Matter of John Gutfreund, Exchange Act Release No. 31554 (Dec. 3, 1992) (sanctions imposed against supervisors at broker-dealer for failing promptly to bring misconduct to attention of the government). See also Federal Sentencing Guidelines § 8C2.5(f) & (g) (organization’s “culpability score” decreases if organization has an effective program to prevent and detect violations of law or if organization reports offense to governmental authorities prior to imminent threat of disclosure or government investigation and within reasonably prompt time after becoming aware of the offense); New York Stock Exchange Rules 342.21 & 351(e) (members and member organizations required to review certain trades for compliance with rules against insider trading and manipulation, to conduct prompt internal investigations of any potentially violative trades, and to report the status and/or results of such internal investigations).
3 In some cases, the desire to provide information to the Commission staff may cause companies to consider choosing not to assert the attorney-client privilege, the work product protection and other privileges, protections and exemptions with respect to the Commission. The Commission recognizes that these privileges, protections and exemptions serve important social interests. In this regard, the Commission does not view a company’s waiver of a privilege as an end in itself, but only as a means (where necessary) to provide relevant and sometimes critical information to the Commission staff. Thus, the Commission recently filed an amicus brief arguing that the provision of privileged information to the Commission staff pursuant to a confidentiality agreement did not necessarily waive the privilege as to third parties. Brief of SEC as Amicus Curiae, McKesson HBOC, Inc., No. 99-C-7980-3 (Ga. Ct. App. Filed May 13, 2001). Moreover, in certain circumstances, the Commission staff has agreed that a witness’ production of privileged information would not constitute a subject matter waiver that would entitle the staff to receive further privileged information.”
III. Output – CIF Meta-Vision Analytics
The following is a screen capture of Meta-Vision analytics on the input text from Section II.
CIF’s NLU logic automatically discovered machine generated hashtags from the source document. Traversing the nodes (MGHT) enables the viewing of the underlying contexts.
IV. Underlying Contexts – Insights from CIF
The following are tables of metrics that depict the insights extracted by CIF Meta-Vision analytics:
The MGHTs shown in the table correspond to the Meta-Vision in Section III. Each of these MGHTs links to context tables that depict contextual excerpts from the source document, self-discovered by CIF’s NLU logic.
The following context tables are from the positive quadrant:
The following context tables are from the negative quadrant:
The following context tables are from the common-negative quadrant:
By examining the context tables for insights, we noted that the NLU algorithm has discovered relevant subjects that form the basis for the cease-and-desist proceeding against Gisela de Leon-Meredith and placed it into the positive quadrant. We further note that CIF’s NLU algorithm documented the subjects that form the basis of the queries regarding the cease-and-desist proceeding and placed it in the negative and common-negative quadrant.
It is worth mentioning that none of these machine generated hashtags are pre-registered with CIF. While many prevailing textual analytics systems require a dictionary and ontology for domain-specific knowledge, CIF does not require the pre-learning of terms that are specific to a domain such as “Fraud”. This use-case provides an example of the utility that organizations can realize through the NLU approach pioneered by SiteFocus – the automation of knowledge discovery with NLU driving instant time to value, catalyzing human creativity, and augmenting cognitive capacity.
Link: Natural Language Understanding (NLU) in Fraud Risk Management – a case study