# Video and Datasets of Roundabout Extracted Real-time Trajectories from a High Resolution Fisheye Camera ## This readme.md file was generated on 20260929 by Ted Morris ## Recommended citation: T. Morris, V. G. Shenoy, V. Morellas, "Video and dataset of Roundabout Extracted Real-time Trajectories from a High Resolution Fisheye Camera", --- # General Information --- ## Title: Video and Datasets of Roundabout Extracted Real-time Trajectories from a High Resolution Fisheye Camera ## Authors 1. Vassilios Morellas, Principal Investigator Department of Electrical and Computer Engineering, University of Minnesota Kenneth H. Keller Hall, Room 4-174 200 Union St. SE morellas@umn.edu ORCID ID: 0000-0002-9198-8442 2. Ted L. Morris, Co-Principal Investigator Department of Computer Science and Engineering, University of Minnesota Kenneth H. Keller Hall, Room 4-192 200 Union St. SE tmorris@umn.edu ORCID ID: 0009-0007-7933-4263 ## Published Date of Release: ## Date of data collection The particular video herein was recorded in 20170311 ## Geographic location The recorded video was collected at 66th and Portland Ave. South, Richfield, MN. ## Funding Sources The work, including roadside camera infrastructure design and deployment, was supported by NSF Cyber Physical Systems program grant #1544887, and the Center for Connected Automated Transportation (CCAT), University of MI, USDOT Grant #69A3551747105. ## Abstract This repository stores corresponding video and output data for the project, *"Roundabout Connected/Automated Vehicle Active Inference Control Strategies to Improve Safety for All Users: Phase-I"*. The video is from a high resolution, 2992x2992 180-degree elevated fisheye camera deployed at the center of an urban arterial 2-lane Roundabout intersection. The footage captures all road user traffic events (Pedestrian, bicycle, public transit, and road vehicle movements) for 30 minutes, at 15 frames per second. The repository includes real-time traffic motion trajectory data from this video. The traffic data includes road user classification type, its temporal real-world location, speed, and heading. In addition, a data conversion python program is provided that also serves as a reference on how to input, manipulate, and save the trajectory data. To further examine, and to generate the output data herein, refer to the open source code git repository, [https://github.com/tedzini/CCAT_Roundabout_Det](https://github.com/tedzini/CCAT_Roundabout_Det). Further information on content within the git code repository is provided below when relevent. --- # Sharing/Access Information --- 1. Licenses/Restrictions * © 2026 Regents of the University of Minnesota. All rights reserved. * The conversion program is shared under the terms and conditions of the GNU Affero General Public License 3.0, License. Further details about the GNU Affero GPL 3.0 license are available in the LICENSE.txt file. 2. The Report,_"Roundabout Connected/Automated Vehicle Active Inference Control Strategies to Improve Safety for All Users: Phase-I"_ which used this data, can be downloaded, free of charge from [https://hdl.handle.net/11299/281279](https://hdl.handle.net/11299/281279) 3. Terms of Use: Data Repository for the U of Minnesota (DRUM) By using these files, users agree to the Terms of Use. https://conservancy.umn.edu/pages/policies/#drum-terms-of-use --- # Data and File Overview --- * **Fri_2017-03-17_153002.avi**: 30 min., 15 fps, 2992x2992 H.264 AVI formatted video recording from the intersection fisheye camera. * **Fri_2017-03-17_153002-SDSM_KF.feather**: Feather formatted Kalman Filtered detected object trajectories output of the first 5 minutes of the video. * **Fri_2017-03-17_153002-SDSM_Raw.feather**: Feather formatted detected object trajectories output of the entire video sequence. * **Fri_2017-03-17_153002-SDSM_KF.csv**: Standard csv formatted text file of Kalman Filtered detected object trajectories output of the first 5 minutes of the video. * **Fri_2017-03-17_153002-SDSM_Raw.csv**: Standard csv formatted text file of detected object trajectories output of the entire video sequence. * **trackdata_convert.py**: Example python program to convert the track data between various file formats, including the raw binary format file of the trajectory data that can be saved from the realtime traffic detection software. A simple example of querying and manipulating the data can also be uncommented within the program. The complete software stack source code is available at [https://github.com/tedzini/CCAT_Roundabout_Det](https://github.com/tedzini/CCAT_Roundabout_Det). * **trackdata_convert.bat**: Windows batch script example usage of **trackdata_convert.py**. * **TM_trackdata.py**: Python module containing structural description of the data files. * **LICENSE.txt**: Disclaimer and terms of use for this data and all associated code herein. ## File Description The Feather file format is a fast, lightweight, and *language-agnostic* binary columnar file format used for storing data frames. It is an open source standard, licensed under Apache License 2.0. For working with this data it suggested to use this format since it is much more efficient. Alternatively, you can use the provided CSV (comma delimited) text file versions. The structural contents of the data is described as follows, with each column header name: * **column 1: 'trackID'**: The assigned object track identifier number assigned by the object detection software. note that all track ID values are integers > 0 * **column 2: 'assoc_TrackID'**: An assigned associated track identifier number determined from the object detection software. If there is more than one tracklet _associated_ with this object track, this ID number will indicate the _original_ track ID number to referrence. * **column 3: 'class'**: Object class name, such as 'car', 'person', 'bus', 'truck'. * **column 4: 'frame'**: The video frame count of the detected trackpoint. Since the video is 15 frames per second, one can directly convert this value to seconds by multiplying it by 1/15*(frame count value). * **column 5: 'lostcnt'**: The frame count that a tracked object was 'lost' at the given sample frame. * **column 6: 'inbounds'**: Indicates if a specific tracked position is within a defined boundary. Please refer to the aformentioned report for a more detailed description. The value of this field is either 'T' for in-bounds, or 'F' when not in-bounds. * **column 7: 'xpos'**: The x-world coordinate, in US feet, of the detected object with respect to an identified intersection center location. The x-axis is parallel to the Minnesota South Easterling stateplane projection. * **column 8: 'ypos'**: The x-world coordinate, in US feet, of the detected object with respect to an identified intersection center location. The y-axis is parallel to the Minnesota South Northing stateplane projection. * **column 9: 'speed'**: The object speed represented in feet/sec. * **column 10: 'heading'**: The object heading in units of radians, with 0-deg pointing North, and +/- 180 degrees pointing South. --- # Methodological Information --- 1. ## Data collection method A panasonic WV-SF481 fisheye camera was configured to capture and stream images at 15 frames per second. The camera was mounted, elevated 35' above the road surface, on a perminant camera pole located at the approximate center of the roundabout intersection. THe video stream was configured with H.264 RTSP protocol, and saved to on-site storage. 2. ## Processing method The trajectory data were created by the trained Deep Learning YOLOv11 model to detect and classify objects in real-time from a calibrated, orthorectified fisheye camera images (J. Kannala and S. S. Brandt, 2006). Individual detected object tracklets were extracted in real-time using ByteTrack object tracking (Y. Zhang et al., 2021) and lost-track recovery as summarized in detail in the referenced report. In short, tracking data are transformed from rectified image coordinates to a real-world geodetic projection state-plane coordinate system, and further associations and filtering are then performed. When a track cannot be recovered ('lost') after a specified amount of time, the track is dropped. Currently this is set to 2.0 seconds, or 30 frames. The complete algorithm implementation, TM_output_detections.py is provided in the github repository. As the real-time traffic object detection and tracking proceed, all tracks were stored in a binary file. The trained object detection YOLO model weights are available in the github repository, [https://github.com/tedzini/CCAT_Roundabout_Det](https://github.com/tedzini/CCAT_Roundabout_Det). 3. ## Analysis software A panda API example is provided within trackdata_convert.py provides an example how to query and manipulate the stored track data. In order to analyze tracks similar to the published report, the [git repo](https://github.com/tedzini/CCAT_Roundabout_Det) provides two programs, Graph_Tracks.py, which references a feather (or binary) formatted track file, and the python program, Overlay_All_Tracks.py, which can be used visualize all identified object tracks overlayed ontop of the rectified fisheye camera video. 4. ## Camera calibration procedure To obtain calibrated optical distortion corrections, grid corner point data extractions were performed in a laboratory setting. An extrinsic site calibration was performed using hand-digitized target points located using Google Earth to extract geodetic latitude/longitude coordinates that can be identified and co-located on a fisheye camera image. The OpenCV library (opencv.org), grid corner point exaction, and PnP 2D-3D solution procedures are used to estimate the intrinsic and extrinsic camera parameters. Further details are summarized in chapter 3 of the reference research report. Camera calibration datasets and calibration software is also available within the github repository, [https://github.com/tedzini/CCAT_Roundabout_Det](https://github.com/tedzini/CCAT_Roundabout_Det). Trajectory accuracies have not be validated by an external measurements. 5. ## Environmental conditions The video file herein was recorded in daylight conditions. 6. ## Staff involved with data collection and processing T. Morris and V. G. Shenoy