The summary statistics by North American Industry Classification System (NAICS) which include: operating revenue (dollars x 1,000,000), operating expenses (dollars x 1,000,000), salaries wages and benefits (dollars x 1,000,000), and operating profit margin (by percent), of motion picture and video distribution (NAICS 512120), annual, for five years of data.

Video Storytelling is a dataset for generating text story/summarization for videos containing social events. It consists of 105 videos from four categories: birthday, camping, Christmas and wedding. For each video, we provide at least 5 human-written stories.

Videos are contained in the .tar file with their corresponding category name.

Text stories are contained in Text.tar.

In each txt file, the first line is the video id. The start and end time (in seconds) of each sentence is also given.

test_id.txt provides the id for videos in the test set

Please cite the following paper if you use the Video Storytelling dataset in your work (papers, articles, reports, books, software, etc):

Video Storytelling: Textual Summaries for Events. J. Li, Y. Wong, Q.Zhao, M. Kankanhalli. IEEE Transactions on Multimedia.

BackgroundJournals are trying to make their papers more accessible by creating a variety of research summaries including graphical abstracts, video abstracts, and plain language summaries. It is unknown if individuals with science, science-related, or non-science careers prefer different summaries, which approach is most effective, or even what criteria should be used for judging which approach is most effective. A survey was created to address this gap in our knowledge. Two papers from Nature on similar research topics were chosen, and different kinds of research summaries were created for each one. Questions to measure comprehension of the research, as well as self-evaluation of enjoyment of the summary, perceived understanding after viewing the summary, and the desire for more updates of that summary type were asked to determine the relative merits of each of the summaries.ResultsParticipants (n = 538) were randomly assigned to one of the summary types. The response of adults with science, science-related, and non-science careers were slightly different, but they show similar trends. All groups performed well on a post-summary test, but participants reported higher perceived understanding when presented with a video or plain language summary (p<0.0025). All groups enjoyed video abstracts the most followed by plain language summaries, and then graphical abstracts and published abstracts. The reported preference for different summary types was generally not correlated to the comprehension of the summaries. Here we show that original abstracts and graphical abstracts are not as successful as video abstracts and plain language summaries at producing comprehension, a feeling of understanding, and enjoyment. Our results indicate the value of relaxing the word counts in the abstract to allow for more plain language or including a plain language summary section along with the abstract.

The MLASK corpus consists of 41,243 multi-modal documents – video-based news articles in the Czech language – collected from Novinky.cz (https://www.novinky.cz/) and Seznam Zprávy (https://www.seznamzpravy.cz/). It was introduced in "MLASK: Multimodal Summarization of Video-based News Articles" (Krubiński & Pecina, EACL 2023). The articles' publication dates range from September 2016 to February 2022. The intended use case of the dataset is to model the task of multimodal summarization with multimodal output: based on a pair of a textual article and a short video, a textual summary is generated, and a single frame from the video is chosen as a pictorial summary.

Each document consists of the following: - a .mp4 video - a single image (cover picture) - the article's text - the article's summary - the article's title - the article's publication date

All of the videos are re-sampled to 25 fps and resized to the same resolution of 1280x720p. The maximum length of the video is 5 minutes, and the shortest one is 7 seconds. The average video duration is 86 seconds. The quantitative statistics of the lengths of titles, abstracts, and full texts (measured in the number of tokens) are below. Q1 and Q3 denote the first and third quartiles, respectively.

/ - / mean / Q1 / Median / Q3 / / Title / 11.16 ± 2.78 / 9 / 11 / 13 / / Abstract / 33.40 ± 13.86 / 22 / 32 / 43 / / Article / 276.96 ± 191.74 / 154 / 231 / 343 /

The proposed training/dev/test split follows the chronological ordering based on publication data. We use the articles published in the first half (Jan-Jun) of 2021 for validation (2,482 instances) and the ones published in the second half (Jul-Dec) of 2021 and the beginning (Jan-Feb) of 2022 for testing (2,652 instances). The remaining data is used for training (36,109 instances).

The textual data is shared as a single .tsv file. The visual data (video+image) is shared as a single archive for validation and test splits, and the one from the training split is partitioned based on the publication date.

All data is part of a pilot study aimed to observe Delta Smelt behavior within enclosures deployed in the wild. The enclosures used were designed by The California Department of Water Resources and UC Davis as part of their ongoing Delta Smelt enclosure deployment studies.
Study 1: This data includes 16 17:43 minute videos of observations of Delta Smelt behavior in a floating field enclosure and an associated .csv file with data generated from the video clips. The videos were recorded using a Gopro Hero 4 Black digital action camera in a field enclosure located in a controlled agriculture pond at the University of California, Davis’ Center for Aquatic Biology and Aquaculture (CABA), Putah Creek Facility, Davis, California, between September 10 and 18, 2019. The data file contains information on the occurrence and duration of alarm behaviors exhibited by Delta Smelt. Four cameras were mounted to the interior of the enclosure at approximately mid height of the enclosure. Viewing area of the cameras extended from approximately the midpoint of the cage bottom to about halfway up the adjacent wall. A video began recording approximately 10 minutes prior to the simulated cage visit and recorded until battery failure resulting in 4, 17:43 minute videos per event. After 17:43 minutes the GoPro camera recorded footage as a separate video file, however they are still part of the same sampling event. Videos were recorded by one camera at a time and consecutive recordings alternated between cameras. Study 2: This data includes 59 10-min video clips of observations of Delta Smelt behavior in a floating field enclosure and an associated .csv file with data generated from the video clips. The videos were recorded using a Gopro Hero 4 Black digital action camera in a field enclosure located in the Sacramento River near Rio Vista, California, between October 12, 2019 and November 2, 2019. The data file contains information on the occurrence and duration of alarm behaviors exhibited by Delta Smelt and the occurrence, duration, and intensity of three types of disturbances: boat noises, noises associated with physical disturbances to the enclosure (e.g., clanking, rattling and banging of the enclosure), and vertical movements of the enclosure associated with wave activity. Two cameras were mounted to the interior of the enclosure at approximately mid height of the enclosure. Viewing area of the cameras extended from approximately the midpoint of the cage bottom to about halfway up the adjacent wall. A video clip of 10-min in duration was recorded at intervals of every 120 min between the hours of 09:00 to 18:00 from 12 October to 02 November 2019. Videos were recorded by one camera at a time and consecutive recordings alternated between cameras.

This dataset contains pretrained models of the CA-SUM network architecture for video summarization, that is presented in our work titled “Summarizing Videos using Concentrated Attention and Considering the Uniqueness and Diversity of the Video Frames”, in Proc. ACM ICMR 2022.

Method overview:

In our ICMR 2022 paper we describe a new method for unsupervised video summarization. To overcome limitations of existing unsupervised video summarization approaches, that relate to the unstable training of Generator-Discriminator architectures, the use of RNNs for modeling long-range frames' dependencies and the ability to parallelize the training process of RNN-based network architectures, the developed method relies solely on the use of a self-attention mechanism to estimate the importance of video frames. Instead of simply modeling the frames' dependencies based on global attention, our method integrates a concentrated attention mechanism that is able to focus on non-overlapping blocks in the main diagonal of the attention matrix, and to enrich the existing information by extracting and exploiting knowledge about the uniqueness and diversity of the associated frames of the video. In this way, our method makes better estimates about the significance of different parts of the video, and drastically reduces the number of learnable parameters. Experimental evaluations using two benchmarking datasets (SumMe and TVSum) show the competitiveness of the proposed method against other state-of-the-art unsupervised summarization approaches, and demonstrate its ability to produce video summaries that are very close to the human preferences. An ablation study that focuses on the introduced components, namely the use of concentrated attention in combination with attention-based estimates about the frames' uniqueness and diversity, shows their relative contributions to the overall summarization performance.

File format:

The “pretrained_models.zip“ file that is provided in the present zenodo page contains a set of pretrained models of the CA-SUM network architecture. After downloading and unpacking this file, in the created “pretrained_models” folder, you will find two sub-directories one per each of the utilized benchmarking datasets (SumMe and TVSum) in our experimental evaluations. Within each of these sub-directories we provide the pretrained model (.pt file) for each data-split (split0-split4), where the naming of the provided .pt file indicates the training epoch and the value of the length regularization factor of the selected pretrained model.

The models have been trained in a full-batch mode (i.e., batch size is equal to the number of training samples) and were automatically selected after the end of the training process, based on a methodology that relies on transductive inference (described in Section 4.2 of [1]). Finally, the data-splits we used for performing inference on the provided pretrained models, and the source code that can be used for training your own models of the proposed CA-SUM network architecture, can be found at: https://github.com/e-apostolidis/CA-SUM.

License and Citation:

These resources are provided for academic, non-commercial use only. If you find these resources useful in your work, please cite the following publication where they are introduced:

E. Apostolidis, G. Balaouras, V. Mezaris, and I. Patras. 2022, “Summarizing Videos using Concentrated Attention and Considering the Uniqueness and Diversity of the Video Frames”, Proc. of the 2022 Int. Conf. on Multimedia Retrieval (ICMR ’22), June 2022, Newark, NJ, USA. https://doi.org/10.1145/3512527.3531404 Software available at: https://github.com/e-apostolidis/CA-SUM

Content Summarization for Video AI Market Outlook

As per our latest research, the global Content Summarization for Video AI market size reached USD 1.36 billion in 2024, reflecting a robust expansion driven by the increasing demand for automated video content analysis and summarization across industries. The market is experiencing a healthy compound annual growth rate (CAGR) of 22.1% and is projected to attain a value of USD 8.93 billion by 2033. This remarkable growth is largely attributed to the rising consumption of video content, the integration of advanced AI technologies in media workflows, and the need for efficient content management solutions in both enterprise and consumer environments.

One of the primary growth factors for the Content Summarization for Video AI market is the exponential surge in video data generated across digital platforms. With the proliferation of streaming services, social media, online education, and enterprise communication tools, organizations are inundated with vast volumes of video content. Manual review and curation of such data are neither scalable nor cost-effective, which has led to the accelerated adoption of AI-powered video summarization solutions. These tools leverage deep learning and natural language processing to extract meaningful insights and generate concise summaries, thereby enhancing content discoverability, improving user engagement, and optimizing storage and bandwidth usage.

Additionally, the rapid advancements in machine learning algorithms and the availability of powerful computational resources have significantly improved the accuracy and efficiency of video AI summarization technologies. TodayÂ’s solutions can recognize complex patterns, detect contextual cues, and even personalize summaries based on user preferences, making them invaluable for sectors such as education, media, and corporate training. The integration of multimodal AI, which combines audio, visual, and textual analysis, further amplifies the capabilities of these platforms, enabling seamless content indexing, real-time highlights generation, and automated compliance monitoring. This technological evolution is fostering widespread adoption among enterprises looking to streamline content workflows and derive actionable intelligence from video assets.

The growing emphasis on accessibility and inclusivity is another critical driver for the Content Summarization for Video AI market. Organizations are increasingly required to provide accessible content for users with diverse needs, including those with hearing or visual impairments. Video AI summarization tools can generate text-based summaries, captions, and alternative formats, ensuring compliance with global accessibility standards and expanding the reach of video content to broader audiences. Furthermore, the rise of remote and hybrid work models has heightened the demand for efficient meeting summarization and knowledge management solutions, further propelling market growth. This trend is especially prominent in sectors such as education, where e-learning platforms are leveraging AI to facilitate personalized and effective learning experiences.

The advent of Video Chaptering AI is revolutionizing how video content is organized and consumed. By automatically segmenting videos into coherent chapters, this technology enhances user navigation and content accessibility, allowing viewers to easily locate and engage with specific sections of interest. This capability is particularly valuable in educational and corporate settings, where users often need to revisit specific topics or presentations. Video Chaptering AI not only improves user experience but also optimizes content management by enabling more efficient indexing and retrieval of video assets. As the demand for personalized and on-demand content grows, the integration of chaptering capabilities is becoming a key differentiator for platforms looking to enhance their video offerings.

From a regional perspective, North America currently dominates the Content Summarization for Video AI market, accounting for the largest share in 2024, driven by the presence of major technology companies, high digital video consumption, and strong investments in AI research and development. However, the Asia Pacific region is rapidly emerging as a high-growth market, fueled by increasing internet penetration, mobile device usage, and the expans

The summary statistics by North American Industry Classification System (NAICS) which include: operating revenue (dollars x 1,000,000), operating expenses (dollars x 1,000,000), salaries wages and benefits (dollars x 1,000,000), and operating profit margin (by percent), of motion picture and video production (NAICS 512110), annual, for five years of data.

This table contains 56 series, with data for years 2006 - 2011 (not all combinations necessarily have data for all years), and was last released on 2015-07-28. This table contains data described by the following dimensions (Not all combinations are available): Geography (14 items: Canada; Prince Edward Island; Nova Scotia; Newfoundland and Labrador ...), North American Industry Classification System (NAICS) (1 items: Motion picture and video production ...), Summary statistics (4 items: Operating revenue; Operating profit margin; Salaries; wages and benefits; Operating expenses ...).

Comprehensive YouTube channel statistics for Summary, featuring 2,450,000 subscribers and 1,349,228,735 total views. This dataset includes detailed performance metrics such as subscriber growth, video views, engagement rates, and estimated revenue. The channel operates in the News-&-Politics category and is based in AE. Track 3,779 videos with daily and monthly performance data, including view counts, subscriber changes, and earnings estimates. Analyze growth trends, engagement patterns, and compare performance against similar channels in the same category.

Summary data from a Calculus II class where students were required to watch an instructional video before or after lecture. Dataset includes gender (1=female; 2=male), vgroup (-1=before lecture; 1=after lecture), binary flag for 26 individual videos (1=watched 80% or more of length of video; 0=not watched), videosum (sum of number of videos watched), final_raw (raw grade student received on cumulative final course exam), sat_math (scaled SAT-Math score out of 800), math_place (institutional calculus readiness score out of 100), watched20 (grouping flag for students who watched 20 or more videos).

AI Video Summarization Market Outlook

According to our latest research, the global AI Video Summarization market size reached USD 1.21 billion in 2024, reflecting the rapid adoption of artificial intelligence technologies across various industries. The market is expected to grow at a robust CAGR of 23.7% during the forecast period, reaching USD 9.06 billion by 2033. This remarkable growth is primarily driven by the increasing demand for efficient content management solutions, the exponential rise in video content generation, and the need for advanced tools to enhance user engagement and operational productivity.

One of the most significant growth factors for the AI Video Summarization market is the overwhelming surge in video data generated across digital platforms. With the proliferation of online streaming services, video-based learning platforms, and corporate training modules, enterprises and individuals face the challenge of managing and extracting relevant insights from vast video libraries. AI-powered video summarization tools offer a transformative solution by automatically generating concise and contextually rich summaries, thereby reducing manual effort and enabling faster content consumption. This capability is particularly valuable for sectors such as media and entertainment, education, and corporate training, where timely access to information and efficient knowledge transfer are crucial for maintaining a competitive edge.

Another key driver propelling the growth of the AI Video Summarization market is the continuous advancements in deep learning, natural language processing, and computer vision technologies. These innovations have significantly improved the accuracy, scalability, and contextual understanding of AI video summarization platforms, making them increasingly reliable for commercial deployment. Organizations are leveraging these tools to automate content tagging, facilitate video search and retrieval, and personalize content delivery based on user preferences. The integration of AI video summarization into enterprise workflows not only streamlines content management but also enhances user engagement, as viewers can quickly access the most relevant segments of lengthy videos without sifting through hours of footage.

Additionally, growing regulatory requirements and the need for compliance in sectors such as healthcare and finance are fostering the adoption of AI Video Summarization solutions. These industries generate large volumes of video data, including telemedicine consultations, training sessions, and surveillance footage, all of which require efficient archiving and documentation. AI-driven summarization tools help organizations adhere to data retention policies, improve auditability, and ensure swift retrieval of critical information. Furthermore, the increasing focus on accessibility, such as providing summarized video content for users with disabilities, is further expanding the market's reach and relevance across diverse user groups.

From a regional perspective, North America continues to dominate the AI Video Summarization market, supported by the presence of leading technology providers, high digital adoption rates, and significant investments in AI research and development. Europe is also witnessing substantial growth, driven by strong regulatory frameworks and the increasing adoption of AI solutions in sectors such as education and healthcare. The Asia Pacific region is emerging as a high-growth market, fueled by the rapid digital transformation of economies like China, India, and Japan, as well as the expanding base of tech-savvy consumers and enterprises. Latin America and the Middle East & Africa, while still nascent, are expected to register steady growth due to increasing digitalization and the rising importance of efficient video content management in these regions.

Component Analysis

The Component segment of the AI Video Summarization market is bifurcated into Software and Services, each playing a pivotal role in the overall ecosystem. The Software component, which encompasses AI algorithms, machine learning models, and user interface platforms, holds the largest market share in 2024. This dominance can be attributed to the growing demand for scalable, automated, and user-friendly solutions that enable organizations to manage and summarize massive video libraries. Software providers are continuously enhancing their offerings with featu

Content Summarization for Video AI Market Outlook

According to our latest research, the global Video AI market size reached USD 2.14 billion in 2024 and is expected to grow at a robust CAGR of 23.2% from 2025 to 2033, reaching a projected value of USD 15.85 billion by 2033. The primary growth driver for the Video AI market is the increasing demand for intelligent video analytics across industries such as media, education, and enterprise, fueled by advancements in deep learning and computer vision technologies.

One of the most significant growth factors propelling the Video AI market is the exponential surge in video content creation and consumption across digital platforms. The proliferation of high-speed internet, affordable smartphones, and the widespread adoption of social media have led to a massive influx of video data. Organizations and individuals require sophisticated tools to analyze, summarize, and manage this content efficiently. Video AI solutions, leveraging state-of-the-art machine learning algorithms, can automatically identify key highlights, generate concise summaries, and extract valuable insights from vast video libraries. This capability not only saves time but also enhances user engagement and content discoverability, making Video AI indispensable for industries reliant on video communication and marketing.

Another crucial driver is the integration of Video AI in educational and corporate settings for e-learning and training purposes. With the global shift towards remote learning and hybrid work environments, educational institutions and enterprises are increasingly utilizing video-based content for lectures, training sessions, and knowledge sharing. Video AI enables automatic indexing, transcription, and summarization of lengthy video materials, allowing learners and employees to access relevant information quickly. This improves knowledge retention, streamlines onboarding processes, and fosters a more personalized learning experience. Moreover, the ability to translate and localize video summaries expands the reach of educational content to a diverse, global audience, further fueling market growth.

The continuous evolution of artificial intelligence and machine learning frameworks has also played a pivotal role in advancing the capabilities of Video AI solutions. Innovations in natural language processing (NLP) and computer vision have significantly improved the accuracy and contextual understanding of video summarization algorithms. This has opened new avenues for applications in sectors such as healthcare, security, and entertainment, where real-time video analysis and summarization are critical. Additionally, the growing availability of cloud-based AI platforms has democratized access to powerful Video AI tools, enabling organizations of all sizes to leverage these technologies without substantial infrastructure investments. The combined impact of these technological advancements and the rising need for efficient content management is expected to sustain the strong growth trajectory of the Video AI market over the forecast period.

From a regional perspective, North America currently dominates the Video AI market, owing to the early adoption of AI technologies, a robust digital infrastructure, and the presence of leading technology companies. However, Asia Pacific is anticipated to witness the fastest growth during the forecast period, driven by rapid digitalization, increasing investments in AI research, and the expanding user base of video-centric platforms in countries like China, India, and Japan. Europe also represents a significant market, with a growing focus on data privacy and compliance shaping the deployment of Video AI solutions. The Middle East & Africa, and Latin America, while currently smaller markets, are expected to see steady growth as digital transformation initiatives gain momentum in these regions. This global expansion is further supported by cross-industry collaborations and strategic partnerships aimed at enhancing the scalability and functionality of Video AI applications.

Component Analysis

The Video AI market is broadly segmented by component into Software and Services. The software segment encompasses a wide range of solutions, including video summarization engines, content indexing tools, and real-time video analytics platforms. These software solutions are designed to automate the process of extracting meaningful insights from v

Edge Video Summarization for Fleets Market Outlook

According to our latest research, the global edge video summarization for fleets market size reached USD 1.16 billion in 2024, with robust demand driven by advancements in artificial intelligence and real-time video analytics. The market is expected to grow at a CAGR of 19.5% from 2025 to 2033, reaching a forecasted market size of USD 5.47 billion by 2033. The primary growth factor fueling this expansion is the increasing need for intelligent fleet management solutions that leverage edge computing to process and summarize vast video data streams for enhanced operational efficiency and safety.

The rapid adoption of edge video summarization technologies in fleet management is primarily propelled by the exponential growth of video surveillance systems installed in commercial vehicles, public transport, and logistics fleets. Traditional cloud-based video analytics often struggle with bandwidth limitations, latency, and privacy concerns, making edge processing a preferred alternative. By enabling real-time video summarization directly on the vehicle, fleet operators can instantly detect incidents, monitor driver behavior, and optimize routes without the need to transmit large volumes of raw footage to centralized servers. This not only significantly reduces operational costs but also ensures compliance with stringent data privacy regulations, particularly in regions with strict data sovereignty laws. The integration of AI-powered summarization algorithms further enhances the accuracy and relevance of insights, empowering fleet managers to make informed decisions swiftly.

Another key driver for the edge video summarization for fleets market is the surge in demand for advanced safety and security solutions across the transportation and logistics sectors. With the escalating incidence of road accidents, cargo theft, and insurance fraud, organizations are increasingly investing in intelligent video analytics that can proactively identify risks and support rapid incident response. Edge video summarization systems enable continuous monitoring and automatic extraction of critical events, allowing stakeholders to review concise, actionable summaries instead of sifting through hours of footage. This capability is especially valuable for large fleets operating in geographically dispersed locations, where centralized video analysis is often impractical. Moreover, the ongoing evolution of connected vehicle ecosystems and the proliferation of IoT devices are further accelerating the adoption of edge-based video analytics, creating new opportunities for market participants.

The market’s robust growth is also attributed to the ongoing digital transformation initiatives within the transportation industry, coupled with government mandates for enhanced fleet safety and compliance. Several countries have introduced regulations requiring commercial fleets to implement video-based driver monitoring and incident reporting systems. This regulatory push, combined with the growing emphasis on operational transparency and customer satisfaction, is compelling fleet operators to embrace edge video summarization solutions. Additionally, the increasing penetration of 5G networks and advancements in edge hardware are enabling seamless deployment of high-performance video analytics at scale. These technological enablers are expected to further catalyze market expansion over the forecast period, as organizations seek to leverage data-driven insights for competitive advantage.

From a regional perspective, North America currently dominates the edge video summarization for fleets market, accounting for the largest revenue share in 2024. The region’s leadership is underpinned by the strong presence of leading technology providers, early adoption of smart fleet management solutions, and stringent regulatory frameworks governing commercial vehicle safety. Europe follows closely, driven by widespread implementation of intelligent transport systems and growing investments in public transit modernization. Meanwhile, the Asia Pacific region is poised for the fastest growth, fueled by rapid urbanization, expanding logistics networks, and government initiatives to enhance road safety. Latin America and the Middle East & Africa are also witnessing increasing adoption, albeit at a comparatively moderate pace, as local stakeholders recognize the benefits of edge video analytics for fleet optimization and risk mitigation.

VideoXum is a large-scale video summarization dataset that contains 14,001 long videos with corresponding human-annotated video and text summaries.

Automatic keyframe detection from videos is an exercise in selecting scenes that can best summarize the content for long videos.

This field activity is part of the effort to map geologic substrates of the Stellwagen Bank National Marine Sanctuary region off Boston, Massachusetts. The overall goal is to develop high-resolution (1:25,000) interpretive maps, based on multibeam sonar data and seabed sampling, showing surficial geology and seabed sediment dynamics. This cruise was conducted in collaboration with the Stellwagen Bank National Marine Sanctuary, and the data collected will aid research on the ecology of fish and invertebrate species that inhabit the region. The Sanctuary's research vessel, R/V Auk, visited 48 locations on Stellwagen Bank at which a customized Van Veen grab sampler (SEABOSS) equipped with a video camera and a CTD was deployed in drift mode to collect sediment for grain-size analysis, video imagery of the seabed, and measurements of water column properties.

This table contains 56 series, with data for years 2006 - 2011 (not all combinations necessarily have data for all years), and was last released on 2015-07-28. This table contains data described by the following dimensions (Not all combinations are available): Geography (14 items: Canada; Prince Edward Island; Nova Scotia; Newfoundland and Labrador ...), North American Industry Classification System (NAICS) (1 items: Motion picture and video production ...), Summary statistics (4 items: Operating revenue; Operating profit margin; Salaries; wages and benefits; Operating expenses ...).

Title-based Video Summarization (TVSum) dataset serves as a benchmark to validate video summarization techniques. It contains 50 videos of various genres (e.g., news, how-to, documentary, vlog, egocentric) and 1,000 annotations of shot-level importance scores obtained via crowdsourcing (20 per video). The video and annotation data permits an automatic evaluation of various video summarization techniques, without having to conduct (expensive) user study.

This dataset contains pretrained models of the PGL-SUM network architecture for video summarization, that is presented in our work titled “Combining Global and Local Attention with Positional Encoding for Video Summarization”, in Proc. IEEE ISM 2021. This work introduces a new method for supervised video summarization, which aims to overcome drawbacks of existing RNN-based summarization architectures that relate to the modeling of long-range frames' dependencies and the ability to parallelize the training process. The proposed PGL-SUM network architecture relies on the use of self-attention mechanisms to estimate the importance of video frames. Contrary to previous attention-based summarization approaches that model the frames' dependencies by observing the entire frame sequence, our method combines global and local multi-head attention mechanisms to discover different modelings of the frames' dependencies at different levels of granularity. Moreover, the utilized attention mechanisms integrate a component that encodes the temporal position of video frames - this is of major importance when producing a video summary. Experiments on two benchmarking datasets (SumMe and TVSum) demonstrate the effectiveness of the proposed model compared to existing attention-based methods, and its competitiveness against other state-of-the-art supervised summarization approaches. File format The provided “pretrained_models.zip“ file contains two sets of pretrained models of the PGL-SUM network architecture. After downloading and unpacking this file, in the created “pretrained_models” folder you will find the following sub-directories: table3_models, table4_models The sub-directory “table3_models“ contains models of the PGL-SUM network architecture that have been trained in a single-batch mode and were manually selected based on the observed summarization performance on the videos of the test set. The average performance of these models (over the five utilized data splits) is reported in Table III of [1]. The sub-directory “table4_models“ contains models of the PGL-SUM network architecture that have been trained in a full-batch mode and were automatically selected after the end of the training process based on the recorded training losses and the application of the designed model selection criterion (described in Section IV.B of our paper). The average performance of these models (over the five utilized data splits) is reported in Table IV of [1]. Each of these sub-directories contains the pretrained model (.pt file), for: Each utilized benchmarking dataset: {SumMe, TVSum} And each utilized data-split: {0, 1, 2, 3, 4} The naming of each of the provided .pt files indicates the training epoch associated with the selected pretrained model. Finally, the data-splits we used for performing inference on the provided pretrained models, and the source code that can be used for training your own models of the proposed PGL-SUM network architecture, can be found at: https://github.com/e-apostolidis/PGL-SUM. License and Citation This dataset is provided for academic, non-commercial use only. If you find this dataset useful in your work, please cite the following publication where it is introduced: [1] E. Apostolidis, G. Balaouras, V. Mezaris, I. Patras, "Combining Global and Local Attention with Positional Encoding for Video Summarization", Proc. 23rd IEEE Int. Symposium on Multimedia (ISM), Dec. 2021. Software available at: https://github.com/e-apostolidis/PGL-SUM Acknowledgements This work was supported by the EU Horizon 2020 programme under grant agreement H2020-832921 MIRROR, and by EPSRC under grant No. EP/R026424/1.