Valuations, P&L, collateral amounts, and reporting data are produced in the order in which each uses the values of an earlier job.

Daytime online processing and end-of-day batch processing

Processing that runs on the spot in response to user actions is called online processing, and processing that runs jobs in groups is called batch processing. In a bank system case, the operation assumes that the previous day's batch processing finishes before online processing starts1.

For derivatives, too, mark-to-market valuation is done separately for two purposes. Traders value their positions during the day to monitor them, and management staff who do not trade value them every day to check the effect on profit and loss (the 1998 examination manual of the US Federal Reserve Board)2. Among the daily jobs, the ones that run together using end-of-day values make up end-of-day batch processing.

"The end of the day" is the point at which the day's trading is closed. Trading positions are normally revalued at least daily, using independent market values as of the close of business3.

Types of end-of-day jobs

The following jobs depend on end-of-day values. The "What the sources say" column shows how the sources describe performing each job daily.

Job What the sources say Values it depends on
Loading end-of-day market data Position valuation uses independent market values as of the close of business. Automated price feeds are used as far as possible3. Prices, rates, and similar data loaded from outside
Revaluation Trading positions are revalued to market at least daily to determine profit and loss3. Market data, the day's trades
Risk calculation and P&L aggregation The risk and profit and loss of positions are reported to non-trading managers at least daily4. Revaluation is the starting point for risk assessment5. Valuations
Collateral calculation For non-cleared OTC derivatives, the institution calculates every day, for each counterparty, the total market value of the transactions and the total market value of variation margin6. Valuations
Preparing reporting data Among the items in trade reporting, the guidelines call the update of valuation and collateral amounts the daily report7. Valuations, collateral amounts
Feed to accounting How often to reconcile the traders' position sheets with items such as the general ledger is set according to the volume and complexity of trading8. Positions, P&L

The procedure for fixing valuations and P&L is covered in Daily Valuation and P&L Sign-Off.

Order and dependencies among jobs

Among the jobs, one relies on the result of another. Revaluation is the starting point for risk assessment5, collateral calculation uses the total market value of transactions6, and the daily report includes valuation and collateral amounts7.

A case in a Bank of Japan document sets completion of the previous day's batch processing at 6 hours before online processing starts. It is an example at one financial institution, and the document does not describe it as a standard10.

Market dataValue and riskCollateralLoad end-of-daymarket dataRecalculateposition valuesAggregate andreport P&Land riskCalculatecollateralby counterpartyPrepare dailyvaluation andcollateral report
Figure 1Dependencies among end-of-day values
Figure description

Market data gives valuations, valuations feed the aggregation of P&L and risk and the calculation of collateral amounts, and collateral amounts feed the daily report. The figure shows dependencies between values, not the order or times of jobs.

Given these dependencies, a late load of market data could delay the valuation, P&L and risk, collateral, and reporting jobs as well.

The cut-off time and time-zone constraints

The examination manual of the US Federal Reserve Board asks institutions to align the time used for valuation. A hedging transaction and the transaction it hedges are valued at prices as of the same time, even when the exchanges' closing times differ. Instruments in the same trading portfolio are valued as of the same time, even when they were traded in different places11.

A time difference constrains processing in foreign exchange settlement. Canceling a payment is subject to the institution's internal processing cycle, its communication networks, and the time differences between branches12.

Preparing for failures and delays

The Comprehensive Guidelines for Supervision of Major Banks, etc. (in Japanese) list points of focus for a bank's contingency plan (a plan for unexpected events). One is whether the plan assumes not only disasters but also system failures caused inside or outside the bank. Another point is whether it assumes adequate risk scenarios, such as a major delay in batch processing (October 2026 version)14.

A Bank of Japan document (March 2007) lists cases of system failures at financial institutions seen in examinations and similar work, together with countermeasures. The cases are not limited to derivatives and include bank deposit and ATM systems. The cases that involve a failure or delay in batch processing are as follows10.

Case What happened Countermeasure the document lists
Conflict between ad hoc and regular batch jobs Both ended abnormally, and because the job could not be started after business start time by design, it could not be rerun and the start of business was greatly delayed. Before ad hoc work, check its effect on parallel work, and state the conditions for a rerun in the work procedure manual.
Abnormal batch end after a CPU upgrade Response time became extremely short, and the checking program, with too few digits, read it as 0 seconds and ended abnormally. The tests had covered only online processing. When changing the system, such as upgrading the CPU, run impact tests on the whole system, including batch processing.
Wrong timing of registering a batch program A program that should have been registered in production after business ended on the business day before the service started was run during the day on the start date, and an error occurred. Create a release procedure manual and set up a system to verify it.
Setting the system start-up time Batch processing finished 6 hours before online processing started, but start-up was 2 hours before, and a failure made the start-up fail, and recovery was not in time for the start of business. Set the start-up time so that, including the recovery time if it fails, it is in time for the start of business.

The daily calculation of market value and collateral amounts (the Cabinet Office Order),6 the daily report of valuation and collateral amounts (the FSA guidelines),7 and preparation for a major delay in batch processing (the supervisory guidelines)14 are described in the regulatory and supervisory sources.

What this article does not cover

The execution order and the start and end times of jobs, batch operations that span Japan and overseas offices, the order in which each office's cut-off time is linked, and regulations that require processing to run overnight are not covered on this page.

Notes

  1. Bank of Japan, Financial System and Bank Examination Department, "事例からみたコンピュータ・システム・リスク管理の具体策" (Concrete measures for computer system risk management, based on cases; our translation) (March 2007), Annex 1, "想定される障害事例と対応策" (Failure cases and countermeasures), the operating procedure case (PDF page 20). It lists online processing and batch processing separately (in Japanese). ↩

  2. Board of Governors of the Federal Reserve System, "Trading and Capital-Markets Activities Manual", Section 2060.1, "Operations and Systems Risk (Back-Office Operations)" (February 1998), page 6. ↩

  3. Board of Governors of the Federal Reserve System, "Trading and Capital-Markets Activities Manual", Section 2100.1, "Financial Performance" (February 1998), page 2, "Daily Mark-to-Market Process." ↩ ↩2 ↩3

  4. Board of Governors of the Federal Reserve System, "Trading and Capital-Markets Activities Manual", Section 2040.1, "Operations and Systems Risk (Management Information Systems)" (February 1998), page 1. ↩

  5. Board of Governors of the Federal Reserve System, "Trading and Capital-Markets Activities Manual", Section 2060.1 (February 1998), page 6. It states that revaluation is the starting point for risk assessment. ↩ ↩2

  6. "金融商品取引業等に関する内閣府令" (Cabinet Office Order on Financial Instruments Business, etc., in Japanese), Article 123, paragraph 1, item 21-10(a). For each counterparty to a non-cleared OTC derivative transaction, the institution calculates every day the total market value of the transactions and the total market value of variation margin. ↩ ↩2 ↩3

  7. Financial Services Agency, "店頭デリバティブ取引等の規制に関する内閣府令第４条第１項で定める作成・保存・報告事項 ガイドライン" (Guidelines on the items to be prepared, retained, and reported under Article 4, paragraph 1 of the Cabinet Office Order on the regulation of OTC derivatives transactions, our translation) (December 2022), Annex 1, item 101, definition of "Action type" (page 26, PDF page 30) (in Japanese). The text refers to the daily report (valuation and collateral amounts), and the actions include Valuation and Collateral. ↩ ↩2 ↩3

  8. Board of Governors of the Federal Reserve System, "Trading and Capital-Markets Activities Manual", Section 2060.1 (February 1998), page 5, "Reconciliations." ↩

  9. Bank of Japan, Financial System and Bank Examination Department, Center for Advanced Financial Technology, "市場リスク管理態勢の整備" (Developing a market risk management framework; our translation) (September 2019), page 37, "フロント部門に対する牽制機能の発揮" (Exercising a check function over the front office) (in Japanese). ↩

  10. Bank of Japan, Financial System and Bank Examination Department, "事例からみたコンピュータ・システム・リスク管理の具体策" (March 2007), Annex 1, "想定される障害事例と対応策" (in Japanese). The conflict between ad hoc and regular batch jobs and the start-up time are on PDF page 20, the CPU upgrade on PDF page 17, and the release of batch programs on PDF pages 13 and 23. ↩ ↩2

  11. Board of Governors of the Federal Reserve System, "Trading and Capital-Markets Activities Manual", Section 2100.1 (February 1998), page 3. ↩

  12. Board of Governors of the Federal Reserve System, "Trading and Capital-Markets Activities Manual", Section 2021.1, "Counterparty Credit Risk and Settlement Risk" (March 1999), page 2. ↩

  13. Board of Governors of the Federal Reserve System, "Trading and Capital-Markets Activities Manual", Section 2021.1 (March 1999), page 2. The 3 a.m. time is the manual's example. ↩

  14. Financial Services Agency, "主要行等向けの総合的な監督指針" (Comprehensive Guidelines for Supervision of Major Banks, etc., in Japanese), III-3-7-1-2 (9) (3) (October 2026), page 267. ↩ ↩2