In the fast-evolving field of artificial intelligence, a new
frontier is emerging: video understanding. Jae Lee, co-founder of
Twelve
Labs, believes that AI models adept at interpreting videos as
well as they do text could unlock groundbreaking applications.
Twelve Labs is pioneering this space by offering functionalities
such as precise searches within video content, clip summarization,
and responses to context-based queries like "When did the
person in the red shirt enter the restaurant?" These
capabilities have garnered investments from major tech firms such
as Nvidia, Samsung, and Intel.
To Lee, a data scientist by training, traditional keyword
searches, which only pull up titles, tags, and descriptions, never
made sense for videos because they fail to access the actual
content of the clips. Frustrated by the inadequacies of existing
solutions, Lee and his team developed a model that maps text to
video content, including actions, objects, and sounds, offering