Processing Video Data in LakeInsight
Demo Content：
1. Read raw video files, extract key frames using Daft, and write to LakeSoul multimodal lakehouse
2. Read and display video frames from LakeSoul
You can access the LakeInsight Demo Environment and open video_demo/video.ipynb to view the demo code.
1. Read video files and extract information using Daft
import logging
logging.disable(logging.CRITICAL)
Use daft.read_video_frames to extract key frames from the video
import daft
from daft import col, DataType
from daft.functions import encode_image
video_path = "/home/maji/data/Projects/multi/video/data/UCF101_subset/train/Basketball/v_Basketball_g01_c01.avi"
# Configure Daft executor
daft.set_runner_ray(noop_if_initialized=True)
df = daft.read_video_frames(
path=video_path,
image_height=480,
image_width=640,
is_key_frame=True,
sample_interval_seconds=1.0,
)
df = df.with_column( "video_path", daft.lit(video_path)).with_column("data", encode_image(col("data"), "JPEG"))
df.show()
Cell Details
Click on a cell to view its full content
2. Write to LakeSoul Multimodal Lakehouse
Use create_table to create a LakeSoul table, then write data
from lakesoul.metadata import create_table
from lakesoul.ray import LakeSoulDatasink
schema = df.schema().to_pyarrow_schema()
create_table(
"video_frames_table",
table_schema=schema,
table_path="/tmp/lakesoul/video_frames_table",
)
ds = df.to_ray_dataset()
sink = LakeSoulDatasink("video_frames_table")
ds.write_datasink(sink)
(pid=181785) PhysicalScan->Project: 0%| | 0.00/1.00 [00:00<?, ?it/s]
3. Read Video Data from Lakehouse
Use ray.data.read_lakesoul() to read the LakeSoul table
Then convert ray.data.Dataset to daft.dataframe
Decode the key frame data into images
import ray
import lakesoul.ray
df = ray.data.read_lakesoul("video_frames_table").to_daft()
from daft import col
from daft.functions import decode_image
df = df.with_column(
"data",
decode_image(col("data"))
)
df.show()
Cell Details
Click on a cell to view its full content