Zentian Ltd v. Apple Inc

I. Executive Summary and Procedural Information

• Parties & Counsel:
  • Plaintiff: Zentian Ltd. (United Kingdom)
  • Defendant: Apple Inc. (California)
  • Plaintiff's Counsel: BRACEWELL LLP; BARTLIT BECK LLP
• Case Identification: 6:22-cv-0122, W.D. Tex., 02/02/2022
• Venue Allegations: Venue is alleged to be proper in the Western District of Texas because Apple maintains a regular and established place of business in Austin, including offices with over 7,000 employees, manufactures certain accused products (Mac Pro) in the district, and has committed acts of infringement there.
• Core Dispute: Plaintiff alleges that Defendant's products featuring the "Hey Siri" voice trigger and other on-device speech recognition functionalities infringe five U.S. patents related to hardware-accelerated speech recognition circuits and methods.
• Technical Context: The technology concerns specialized hardware architectures designed to make speech recognition faster and more power-efficient on mobile and other power-constrained electronic devices.
• Key Procedural History: The complaint alleges that Zentian met with Apple's Silicon Engineering Group in 2006 and its Patent Acquisitions department in 2015 to discuss its technology and patent portfolio, which may be relevant to the plaintiff's allegations of willful infringement.

Case Timeline

Date	Event
2000-01-01	Zentian founded
2002-02-04	Earliest Priority Date for '319 and '140 Patents
2004-09-14	Earliest Priority Date for '277, '377, and '789 Patents
2006-01-01	Zentian meets with Apple's Silicon Engineering Group
2009-09-08	U.S. Patent No. 7,587,319 Issued
2011-01-01	Siri first introduced by Apple
2011-07-12	U.S. Patent No. 7,979,277 Issued
2014-09-01	Apple introduces the "Hey Siri" feature
2015-01-01	Zentian meets with Apple's Patent Acquisitions department
2018-08-28	U.S. Patent No. 10,062,377 Issued
2020-11-17	U.S. Patent No. 10,839,789 Issued
2021-04-06	U.S. Patent No. 10,971,140 Issued
2022-02-02	Complaint Filed

II. Technology and Patent(s)-in-Suit Analysis

U.S. Patent No. 7,979,277 - "Speech Recognition Circuit and Method"

• Patent Identification: U.S. Patent No. 7,979,277, "Speech Recognition Circuit and Method," issued on July 12, 2011 Compl. ¶42

The Invention Explained

• Problem Addressed: Conventional speech recognition systems in the early 2000s were software-based, making them too slow and power-intensive for effective use on mobile devices like smartphones, particularly in noisy environments Compl. ¶5 The patent family addresses the limitations of power, space, and cost in mobile electronic devices that made effective speech recognition difficult '377 Patent, col. 1:21-49
• The Patented Solution: The invention proposes a specific hardware circuit architecture that improves performance and power efficiency by structuring the speech recognition process into a pipeline of distinct hardware components Compl. ¶46 As recited in the claims, this includes an audio front end, a calculating means, and a search stage that are connected to enable a "pipelined data flow" Compl. ¶82 The patent family specification illustrates this as a sequence of functional blocks, including a "Front End," "Distance Calculation," and "Search Stage" '377 Patent, Fig. 16
• Technical Importance: This hardware-centric, pipelined approach was intended to solve the power and speed constraints that had previously limited complex, on-device speech recognition features on mobile products (Compl. ¶¶6; Compl. ¶37).

Key Claims at a Glance

• The complaint asserts independent claims 14 (a circuit), 15 (a method), and 16 (a non-transitory storage medium) Compl. ¶81
• The essential elements of independent claim 14 are:
  • An audio front end for calculating a feature vector from an audio signal.
  • A calculating means for calculating a distance indicating the similarity between the feature vector and a predetermined acoustic state of an acoustic model.
  • A search stage for using the calculated distances to identify words within a lexical tree.
  • Wherein the audio front end, calculating means, and search stage are connected to enable pipelined data flow.
• The complaint reserves the right to assert additional claims Compl. ¶115

U.S. Patent No. 7,587,319 - "Speech Recognition Circuit Using Parallel Processors"

• Patent Identification: U.S. Patent No. 7,587,319, "Speech Recognition Circuit Using Parallel Processors," issued on September 8, 2009 Compl. ¶47

The Invention Explained

• Problem Addressed: The computationally intensive nature of the search process in conventional software-based speech recognition, which attempts to find the most likely words from a given vocabulary Compl. ¶51
• The Patented Solution: The invention describes a hardware architecture that uses a plurality of "lexical tree processors" connected in parallel to distribute the word recognition task Compl. ¶51 The patent family describes this as a system with parallel processors for performing a lexical tree search, controlled by a controller '140 Patent, abstract
• Technical Importance: This parallel hardware architecture was designed to enable faster, real-time processing of more complex and robust speech models by dividing the computational workload Compl. ¶51

Key Claims at a Glance

• The complaint asserts independent claim 46 Compl. ¶123
• The essential elements of independent claim 46 are:
  • An input buffer receiving processed speech parameters.
  • A plurality of lexical memories containing, in combination, complete lexical data for word recognition.
  • A plurality of processors connected in parallel to the input buffer, arranged in groups, with each group connected to a lexical memory.
  • A control processor controlling each processor to process speech parameters using partial lexical data from a respective lexical memory.
  • A results memory for storing the results of the processing.
• The complaint reserves the right to assert additional claims Compl. ¶147

U.S. Patent No. 10,971,140 - "Speech Recognition Circuit Using Parallel Processors"

• Patent Identification: U.S. Patent No. 10,971,140, "Speech Recognition Circuit Using Parallel Processors," issued April 6, 2021 Compl. ¶52
• Technology Synopsis: The '140 Patent is in the same family as the '319 Patent. It describes a speech recognition circuit with processor clusters configured to generate an initial score from an audio sample. This initial score is then used to determine whether to continue with more intensive processing, using a larger amount of model data, to generate a final score Compl. ¶56
• Asserted Claims: The complaint asserts independent claim 1 Compl. ¶155
• Accused Features: The "Hey Siri" feature is accused of infringing, specifically its two-stage detection process. This allegedly involves using a low-power processor (AOP) with a "small DNN" to generate an initial score, and if a threshold is met, waking a more powerful main processor with a "larger DNN" to determine a final score Compl. ¶¶165-166

U.S. Patent No. 10,062,377 - "Distributed Pipelined Parallel Speech Recognition System"

• Patent Identification: U.S. Patent No. 10,062,377, "Distributed Pipelined Parallel Speech Recognition System," issued August 28, 2018 Compl. ¶57
• Technology Synopsis: The '377 Patent is in the same family as the '277 Patent. It describes a speech recognition system comprising a series of distinct programmable devices. A first device calculates a feature vector, a second calculates distances to an acoustic model, and a third identifies words using techniques like Hidden Markov Models or Neural Networks, followed by a search stage Compl. ¶61
• Asserted Claims: The complaint asserts independent claim 1 and dependent claims 2-6 Compl. ¶184
• Accused Features: Apple's on-device speech recognition architecture is accused of infringement. This includes the alleged use of a first programmable device (the AOP) to calculate feature vectors, a second programmable device (e.g., Neural Engine or GPU) to run a DNN acoustic model, and a third programmable device (an application processor) to perform the final search stage Compl. ¶¶190-201

U.S. Patent No. 10,839,789 - "Speech Recognition Circuit and Method"

• Patent Identification: U.S. Patent No. 10,839,789, "Speech Recognition Circuit and Method," issued November 17, 2020 Compl. ¶62
• Technology Synopsis: The '789 Patent is in the same family as the '277 Patent. It describes an "acoustic coprocessor" for processing audio data. The coprocessor includes an interface to receive feature vectors, a calculating apparatus for determining distances to an acoustic model, and a second interface to send the calculated distances Compl. ¶66
• Asserted Claims: The complaint asserts independent claim 10 Compl. ¶223
• Accused Features: For certain MacBook Pro models, the complaint alleges that the Intel Gaussian & Neural Accelerator (GNA) is the infringing acoustic coprocessor Compl. ¶¶224-226 In other products, the Always-On Processor (AOP) is implicated as the coprocessor that performs initial processing before passing data to a main processor Compl. ¶165

III. The Accused Instrumentality

• Product Identification: The complaint identifies Apple's iPhone, iPad, Apple Watch, AirPods, HomePod, HomePod Mini, Apple TV set-top box, and Mac computers as the "Accused Products" Compl. ¶14
• Functionality and Market Context:
  • The complaint focuses on three key functionalities: (1) "Wake phrase detection" for "Hey Siri"; (2) Dictation of text messages and documents; and (3) On-device speech recognition for Siri user requests Compl. ¶73
  • The core technical operation is described as a multi-stage, hardware-assisted process. An "Always On Processor (AOP)," described as a "small, low-power auxiliary processor," continuously listens for the wake phrase Compl. ¶88 Upon hearing a potential trigger, the AOP calculates feature vectors (MFCCs) and processes them with a "small version of the acoustic model (DNN)" (Compl. ¶¶88; Compl. ¶165). If the resulting score exceeds a threshold, the AOP "wakes up the main processor" (such as the Apple Neural Engine or GPU), which then "analyzes the signal using a larger DNN" for more accurate recognition Compl. ¶134 Compl. ¶165 A diagram in the complaint illustrates this two-tiered system, separating "Low-compute (always on)" functions from "More Accurate (main processor)" functions Compl. p. 21
  • Siri is alleged to be a "core feature built into almost every device Apple sells" and "critical to the unique user-device interface that drives Apple's market share," reportedly handling over 25 billion requests per month Compl. ¶70

IV. Analysis of Infringement Allegations

U.S. Patent No. 7,979,277 Infringement Allegations

Claim Element (from Independent Claim 14)	Alleged Infringing Functionality	Complaint Citation	Patent Citation
an audio front end for calculating a feature vector from an audio signal...	The "Always On Processor (AOP)" is alleged to be the audio front end, which calculates feature vectors via Mel Frequency Cepstrum Coefficient (MFCC) computations from the microphone signal.	¶88	col. 11:50-55
calculating means for calculating a distance indicating the similarity between a feature vector and a predetermined acoustic state of an acoustic model	The "calculating means" is alleged to be a Deep Neural Network (DNN) acoustic model, which runs on a main processor (e.g., Neural Engine or GPU) and converts the feature vectors into a probability distribution over speech sound classes.	¶¶91-93	col. 12:5-9
a search stage for using said calculated distances to identify words within a lexical tree...	The "search stage" is alleged to be the final temporal integration computation, which Apple describes as an "application of dynamic programming" based on Hidden Markov Models (HMMs) to identify words.	¶¶96; ¶98	col. 12:16-24
wherein said audio front end, said calculating means, and said search stage are connected to each other to enable pipelined data flow	The complaint alleges data flows in a pipelined manner from the AOP (front end) to a second processor like the Neural Engine/GPU (calculating means), and then to a search stage that runs on either the AOP or a main application processor. A diagram in the complaint is used to illustrate this alleged flow.	¶104	col. 23:5-12

• Identified Points of Contention:
  • Scope Questions: Claim 14 recites "calculating means," which is a means-plus-function limitation under 35 U.S.C. § 112, paragraph 6. Its scope is defined by the corresponding structure disclosed in the patent's specification and its equivalents. A central question will be whether the structure disclosed in the '277 patent family (e.g., hardware for performing Mahalanobis distance calculations) is structurally equivalent to Apple's accused Deep Neural Network (DNN) running on a Neural Engine or GPU.
  • Technical Questions: The complaint alleges a pipelined architecture but suggests the "search stage" may run on either the AOP or a main processor Compl. ¶104 The precise hardware mapping and data-flow path between the alleged "audio front end" (AOP), "calculating means" (main processor DNN), and "search stage" will be a key factual dispute for determining if the "connected to each other" limitation is met. The complaint includes a speech recognizer diagram, annotated to show the distinct stages, which alleges the "stream of feature vectors" from the audio front end feeds the distance calculation block, which in turn feeds the search stage Compl. p. 22

U.S. Patent No. 7,587,319 Infringement Allegations

Claim Element (from Independent Claim 46)	Alleged Infringing Functionality	Complaint Citation	Patent Citation
an input buffer receiving processed speech parameters	The "Frame Buffer" located in the AOP is alleged to be the input buffer, as it stores processed speech parameters (feature vectors from MFCC computations) before they are sent to the main processor.	¶127	col. 10:1-2
a plurality of lexical memories...each lexical memory containing part of said complete lexical data	The combination of non-volatile FLASH memory and the hierarchical cache memories (e.g., L2, L3) are alleged to be the "plurality of lexical memories," as they collectively store the complete speech recognition models (lexical data).	¶¶128-130	col. 10:3-6
a plurality of processors connected in parallel to said input buffer...said processors being arranged in groups of processors, each group of processors being connected to a lexical memory	The multi-core application processors in Apple's SoCs (e.g., A14/A15) are alleged to be the plurality of processors. The complaint alleges these are arranged in clusters (groups) and connected to the cache hierarchy (lexical memory) and receive data from the input buffer. Die shot photos are provided as evidence of these clusters.	¶¶131-133	col. 10:7-13
a control processor controlling each processor to process said speech parameters using partial lexical data...	The AOP is alleged to be the control processor. It is alleged to control the main processor by running a small detector and, upon exceeding a threshold, "waking up the main processor" to analyze the signal with a larger dataset.	¶134	col. 10:14-18
a results memory storing the results of the processing of the speech parameters from said processors	The SDRAM in the Accused Products is alleged to be the results memory, as it is the main working memory of the system and would store the output of the speech processing.	¶¶136-137	col. 10:19-21

• Identified Points of Contention:
  • Scope Questions: The infringement theory relies on the AOP's "wake up" signal satisfying the "controlling each processor" limitation. A key question of claim scope will be whether "controlling" requires continuous management and instruction, or if a single gating action (waking the processor) is sufficient to meet the claim's requirement.
  • Technical Questions: The complaint identifies general-purpose memory components like FLASH, SDRAM, and CPU caches as the claimed "lexical memories" and "results memory." A factual question will be whether these components are used in the specific manner required by the claims or if their function is sufficiently distinct from the patented architecture. The complaint provides die photos of Apple's A14 and A15 SoCs to support its allegations of processor clusters Compl. p. 34

V. Key Claim Terms for Construction

• For the '277 Patent:

  • The Term: "calculating means" (from claim 14)
  • Context and Importance: This is a means-plus-function term governed by 35 U.S.C. § 112, para. 6. Its construction will be dispositive for infringement, as it defines the core computational element of the claim. The dispute will center on whether Apple's accused DNNs are structurally equivalent to the specific algorithm(s) and/or circuit(s) disclosed in the patent's specification for performing the function of "calculating a distance."
  • Intrinsic Evidence for Interpretation:
    • Evidence for a Broader Interpretation: A plaintiff may argue that the specification's description of the "Distance Calculation" block is primarily functional '377 Patent, col. 12:5-15 and that any structure performing that function, including a DNN, is an equivalent.
    • Evidence for a Narrower Interpretation: A defendant may point to specific embodiments in the specification, such as the detailed description of a Mahalanobis Distance calculation '377 Patent, col. 24:9-12, to argue that the claim is limited to that specific algorithm and its narrow structural equivalents, potentially excluding the different architecture of a DNN.

• For the '319 Patent:

  • The Term: "control processor" (from claim 46)
  • Context and Importance: The infringement allegation hinges on the AOP acting as the "control processor." Practitioners may focus on this term because the accused functionality is a "wake up" signal, and the case may turn on whether this action constitutes "controlling" in the context of the patent.
  • Intrinsic Evidence for Interpretation:
    • Evidence for a Broader Interpretation: A plaintiff may argue the plain and ordinary meaning of "control" includes gating or enabling a process. The patent describes a "control processor controlling each processor to process said speech parameters" '140 Patent, col. 10:14-16, which could be read broadly to include an initiation command.
    • Evidence for a Narrower Interpretation: A defendant may argue that the term, read in light of the specification, requires more granular, ongoing management of the parallel processors. The abstract of the '140 Patent (same family) states the controller "controls the lexical tree processors to process lexical trees identified in the results memory," suggesting a function more complex than a simple wake-up call '140 Patent, abstract

VI. Other Allegations

• Indirect Infringement: The complaint alleges Apple induces infringement by providing "directions, instruction manuals, guides, and/or other materials that encourage and facilitate infringing use by others" Compl. ¶107 It also alleges contributory infringement, stating that Apple sells components that are a material part of the invention and are not a staple article of commerce suitable for substantial non-infringing use Compl. ¶109
• Willful Infringement: The willfulness allegation is based on alleged pre-suit knowledge. The complaint cites meetings in 2006 and 2015 where Zentian representatives allegedly discussed the patented technology with Apple personnel, giving Apple notice of the patents Compl. ¶¶39-40 Compl. ¶74 Compl. ¶111 This is alleged to make the infringement "deliberate and intentional" Compl. ¶75

VII. Analyst's Conclusion: Key Questions for the Case

• A core issue will be one of structural equivalence under 35 U.S.C. § 112(f): for the '277 patent family, is Apple's accused Deep Neural Network (DNN) architecture a structural equivalent to the specific "calculating means" disclosed in the patent's specification, or is there a fundamental difference in structure and operation that places it outside the claim's scope?
• A second central question will be one of definitional scope: for the '319 patent family, can the term "control processor" be construed to cover a low-power auxiliary processor (the AOP) that performs a threshold-based "wake-up" function, or does the claim, read in light of the specification, require a more active and continuous management role over the parallel processors?
• A key evidentiary question will be one of technical mapping: does the actual, verifiable data flow and division of labor between Apple's various processing units-the AOP, Neural Engine, GPU, and CPU cores-map directly onto the discrete, serially connected "audio front end," "calculating means," and "search stage" required by the claims of the '277 patent family, or is there a functional mismatch in the system's architecture?