Commons talk:Video2commons

	SpBot archives all sections tagged with {{Section resolved|1=~~~~}} after 1 day and sections whose most recent comment is older than 90 days. For the archive overview, see Special:PrefixIndex/Commons talk:Video2commons/Archive.

Before you file a report, please check whether the following list already includes the problem you have.

Known bugs and common errors

1. Subtitles are not imported https://github.com/toolforge/video2commons/issues/148

   Since circa 2020, subtitles have not been automatically imported even if the option is selected. Unknown cause, possibly bug in codes. Previous discussions: Commons_talk:Video2commons/Archive_2#Youtube_subtitle_not_imported Commons_talk:Video2commons/Archive_2#Not_importing_subtitles_from_youtube.--RZuo (talk) 17:04, 1 January 2023 (UTC)

Large uploads smoother

https://phabricator.wikimedia.org/T358308

thanks to User:CGoubert-WMF, User:HNowlan (WMF) and User:Bawolff, large uploads are now probably smoother. maybe v2c also benefits from this? how's your experience recently? RZuo (talk) 21:13, 27 April 2024 (UTC)

I should mention that I'm still aware of at least two issues that might affect big uploads (interuption during deployments, and false positives when scanning for bad files) however both should happen a lot less then the previous issue that was recently fixed, so hopefully things are working smoother now even if not perfect. Bawolff (talk) 22:53, 27 April 2024 (UTC)

File:Discovering Tenggol- Aerial Views and Submerged Delights in 4K.webm 3.28 gb, successful in 1 try. RZuo (talk) 11:51, 28 April 2024 (UTC)

File:Data Center Tour & Technical Deep Dive into the Power, Data and Cooling Infrastructure!.webm 3.31 gb success in 1 try. RZuo (talk) 14:42, 28 April 2024 (UTC)

@Bawolff: Hi, I got a message An exception occurred: MaybeEncodingError: b'(\'\\\'PicklingError("Can\\\\\\\'t pickle <class \\\\\\\'video2commons.exceptions.TaskError\\\\\\\'>: import of module \\\\\\\'video2commons.exceptions\\\\\\\' failed")\\\'\', \'"(1, <ExceptionInfo: TaskError(\\\'Sorry, but files larger than 4GB can not be uploaded even with server-side uploading. This task may need manual intervention.\\\')>, None)"\')'. However the MP4 file is only 1.252 GB. So how is it possible to get larger than 4 GB here? Thanks, Yann (talk) 10:45, 14 May 2024 (UTC)

Its hard to know without looking at the file (and possibly knowing more about mp4 internals then i do), but for some files there can be transcoding options that well normally work well might work terribly for some specific file. As an aside as far as 4gb limit goes on video2commons, i submitted [1] but its waiting on one of the video2commons maintainers to review it (i think that is @Zhuyifei1999 ) Bawolff (talk) 20:43, 14 May 2024 (UTC)

@Bawolff: Thanks for your answer. It is not the only problem. See the screenshot I posted below. Yann (talk) 22:10, 14 May 2024 (UTC)

The problems below all sound like variations on out-of-memory errors (the cannot allocate memory error very obviously looks like OOM, but the errors related about processes being killed sound like they could also be OOM related errors). I don't know enough how video2commons works to say more than that. Perhaps the tool in question needs to be modified to do less things in parallel. Bawolff (talk) 00:19, 15 May 2024 (UTC)

@Bawolff: I wonder if I should open bug reports, and with what information? Yann (talk) 08:15, 15 May 2024 (UTC)

I got the following while trying to upload https://archive.org/details/where-the-north-begins-1923-by-chester-m.-franklin to File:Where the North Begins (1923, 1080p) by Chester M. Franklin.webm: "An exception occurred: MaybeEncodingError: b'(\'\\\'PicklingError("Can\\\\\\\'t pickle <class \\\\\\\'video2commons.exceptions.TaskError\\\\\\\'>: import of module \\\\\\\'video2commons.exceptions\\\\\\\' failed")\\\'\', \'"(1, <ExceptionInfo: TaskError(\\\'Sorry, but files larger than 4GB can not be uploaded even with server-side uploading. This task may need manual intervention.\\\')>, None)"\')'".   — 🇺🇦Jeff G. ツ please ping or talk to me🇺🇦 17:25, 7 July 2024 (UTC)

It does not import subtitles!

Example that has subtitles at source and when downloaded

When having subtitles checked it usually does not import subtitles, or at least often.

For example, try turning on subtitles for the video on the right which has subtitles at the source. Here is another example since I think the uploader had import subtitles checked (note that in this case the subtitles are set on a wrong language at the source which is rare). Could this please be fixed as soon as possible so people don't need to upload many videos again to have the subtitles working? It would be best if one could easily add subtitles to the video file itself so that they are embedded in the video but for now this is only about the subtitles existing at the source.

There also is an issue that subtitles are not shown for videos where the subtitle was successfully imported using the tool (you can download the video and watch the video with subtitles); create an issue about that here: phab:T368298. Prototyperspective (talk) 17:22, 24 June 2024 (UTC)

#top "Before you file a report, please check whether the following list already includes the problem you have." RZuo (talk) 19:40, 24 June 2024 (UTC)

Ups, missed that part. Was about to delete this section but seeing how this is still an issue after ca 4 years, the issue has been created over 2 years ago, and there not being much details in the github issue maybe it should stay. Getting the subtitle downloaded is just making sure the right yt-dlp command is used so if that is the problem it should be quick and simple to fix. Also maybe it has to do with the phab issue about VTT subtitles, should there also be a phabricator issue about it? From the code issue it seems like it could have to do with language codes, could somebody take a look at it? Prototyperspective (talk) 21:17, 24 June 2024 (UTC)

Cannot upload age-restricted videos

I get the following error when I try to upload videos that are age-restricted:

An exception occurred: DownloadError: b'ERROR: unable to download video data: HTTP Error 403: Forbidden'

Is there a work-around? FunnyMath (talk) 17:58, 11 July 2024 (UTC)

@FunnyMath: I don't suppose we could wait til the bot is 18.   — 🇺🇦Jeff G. ツ please ping or talk to me🇺🇦 08:00, 13 July 2024 (UTC)

@Jeff G.: Maybe the bot can be accompanied by another bot that is 18 or older ;)

Anyways, it's not an urgent issue. I'm using yt-dlp to bypass the age-restriction. I'm uploading some public domain Korean War videos that are not child-friendly, so that's why. My issue might be too niche to merit a fix; I would guess not too many public domain or CC-BY videos are age-restricted. FunnyMath (talk) 09:11, 13 July 2024 (UTC)

What's the option you're using to bypass the restriction? We can simply include the option in video2commons as well. vip (talk) 16:20, 19 August 2024 (UTC)

I've got the same problem too. – Illegitimate Barrister (talk • contribs), 17:41, 17 July 2024 (UTC)

AV1 encoding feature request

I think that video2commons should keep up with the times and have the option to convert videos to the AV1 format. Also, v2c should have the option to transcode only a specified section of a source video if, for example, the video contains something copyrightable like an intro animation. prospectprospekt (talk) 04:02, 23 July 2024 (UTC)

+1. phab:T209437. Yann (talk) 16:34, 23 July 2024 (UTC)

Agreed. I'll check into that after the ffmpeg/gentoo update (see below). vip (talk) 16:15, 19 August 2024 (UTC)

API for this tool

Hi. Would it be possible to document the API so that I can send task jobs from my workflow? -- DaxServer (talk) 18:59, 14 August 2024 (UTC)

Hello @DaxServer the API is implemented here: https://github.com/toolforge/video2commons/blob/master/video2commons/frontend/api.py

There is no documentation AFAIK but it's quite easy to understand, I managed to do it in my own tool: https://gitlab.wikimedia.org/toolforge-repos/spacemedia/-/blob/main/src/main/java/org/wikimedia/commons/donvip/spacemedia/service/wikimedia/Video2CommonsService.java vip (talk) 16:10, 19 August 2024 (UTC)

Thanks for the info @Don-vip I'll look into it! -- DaxServer (talk) 16:39, 19 August 2024 (UTC)

This could be useful to build a tool to for example upload all CCBY videos from a channel. This could be useful to for example upload all free audio files of a musician channel. Please link such tools in the V2C documentation. Prototyperspective (talk) 12:17, 20 August 2024 (UTC)

Issue 212

verbatim nowikid: Please file an issue in GitHub: Traceback (most recent call last): File "/data/project/video2commons/www/python/venv/lib/python3.11/site-packages/flask/app.py", line 880, in full_dispatch_request rv = self.dispatch_request() ^^^^^^^^^^^^^^^^^^^^^^^ File "/data/project/video2commons/www/python/venv/lib/python3.11/site-packages/flask/app.py", line 865, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/project/video2commons/www/python/src/../../../video2commons/frontend/app.py", line 134, in main return render_template( ^^^^^^^^^^^^^^^^ File "/data/project/video2commons/www/python/venv/lib/python3.11/site-packages/flask/templating.py", line 150, in render_template return _render(app, template, context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/project/video2commons/www/python/venv/lib/python3.11/site-packages/flask/templating.py", line 131, in _render rv = template.render(context) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/project/video2commons/www/python/venv/lib/python3.11/site-packages/jinja2/environment.py", line 1304, in render self.environment.handle_exception() File "/data/project/video2commons/www/python/venv/lib/python3.11/site-packages/jinja2/environment.py", line 939, in handle_exception raise rewrite_traceback_stack(source=source) File "/data/project/video2commons/video2commons/frontend/templates/main.min.html", line 1, in top-level template code {% extends "base.min.html" %} {% block jscss %} {% if loggedin %}<script src="//tools-static.wmflabs.org/cdnjs/ajax/libs/jquery/2.2.0/jquery.min.js"></script><script src="//tools-static.wmflabs.org/cdnjs/ajax/libs/twitter-bootstrap/3.3.6/js/bootstrap.min.js"></script><script src="//tools-static.wmflabs.org/cdnjs/ajax/libs/nunjucks/3.0.1/nunjucks-slim.min.js"></script><script src="//tools-static.wmflabs.org/cdnjs/ajax/libs/qs/6.5.2/qs.min.js"></script><script src="//tools-static.wmflabs.org/cdnjs/ajax/libs/blueimp-file-upload/9.12.5/js/vendor/jquery.ui.widget.js"></script><script src="//tools-static.wmflabs.org/cdnjs/ajax/libs/blueimp-file-upload/9.12.5/js/jquery.fileupload.min.js"></script><link href="//tools-static.wmflabs.org/cdnjs/ajax/libs/blueimp-file-upload/9.12.5/css/jquery.fileupload.css" rel="stylesheet">{% if config.socketio_uri %}<script src="{{ config.socketio_uri }}/socket.io.js"></script>{% endif %}<script src="config"></script><script src="i18n/{{ lang() }}"></script><script src="static/templates.min.js"></script><script src="static/video2commons.min.js"></script>{% endif %} {% endblock %} {% block content %} {% if loggedin %}<noscript>{{ _('JavascriptRequired') }}</noscript>{% else %}<form method="get" action="{{ url_for('loginredirect') }}"><center><input class="btn btn-primary btn-success btn-lg" type="submit" value="{{ _('loginOnCommons') }}"></center><input type="hidden" name="returnto" value="{{ request.url }}"></form>{% endif %} {% endblock %} ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/project/video2commons/video2commons/frontend/templates/base.min.html", line 1, in top-level template code <!DOCTYPE html><html xmlns="http://www.w3.org/1999/xhtml" dir="{% if lang() is rtl %}rtl{% else %}ltr{% endif %}" lang="{{ lang() }}"><head><meta http-equiv="Content-type" content="text/html;charset=UTF-8"><title>video2commons</title><link href="//tools-static.wmflabs.org/cdnjs/ajax/libs/twitter-bootstrap/3.3.6/css/bootstrap.min.css" rel="stylesheet">{% if lang() is rtl %}<!--<link href="//tools-static.wmflabs.org/cdnjs/ajax/libs/bootstrap-rtl/3.1.2/css/bootstrap-rtl.min.css" rel="stylesheet">--><link href="static/externals/bootstrap-rtl.min.css" rel="stylesheet">{% endif %}<link href="//tools-static.wmflabs.org/cdnjs/ajax/libs/github-fork-ribbon-css/0.1.1/gh-fork-ribbon.min.css" rel="stylesheet">{% block jscss %}{% endblock %}<style>body{padding-top:60px}.github-fork-ribbon{background-color:#333}</style></head><body><div class="navbar navbar-default navbar-fixed-top"><div class="container-fluid"><div class="navbar-header"><a class="navbar-brand" href="#">video2commons</a></div><ul class="nav navbar-nav navbar-{% if lang() is rtl %}left{% else %}right{% endif %}"><li><a href="https://commons.wikimedia.org/w/index.php?title=Special:MyLanguage/Commons:Video2commons&uselang={{ lang() }}"><span class="glyphicon glyphicon-book"></span> {{ _('help') }}</a></li>{% if loggedin %}<li><a href="{{ url_for('logout') }}"><span class="glyphicon glyphicon-log-out"></span> {{ _('logout') }}</a></li>{% endif %}</ul></div></div><div class="container" id="content">{% block content %}{% endblock %}</div><div class="github-fork-ribbon-wrapper {% if lang() is rtl %}left{% else %}right{% endif %}-bottom"><div class="github-fork-ribbon"><a href="https://github.com/toolforge/video2commons">{{ _('forkOnGitHub') }}</a></div></div></body></html> ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/project/video2commons/www/python/src/../../../video2commons/frontend/i18n.py", line 103, in translate return get(getlanguage()).get(key, '<' + key + '>') ^^^^^^^^^^^^^^^^^^ File "/data/project/video2commons/www/python/src/../../../video2commons/frontend/i18n.py", line 56, in get return json.loads(redisconnection.get(i18nkey)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/__init__.py", line 339, in loads raise TypeError(f'the JSON object must be str, bytes or bytearray, ' TypeError: the JSON object must be str, bytes or bytearray, not NoneType   — 🇺🇦Jeff G. ツ please ping or talk to me🇺🇦 00:20, 19 August 2024 (UTC)

Hello Jeff, reporting these issues on GitHub is enough, no need to duplicate the message here :) vip (talk) 16:14, 19 August 2024 (UTC)

Maintenance: FFmpeg/Gentoo update

Hello, when trying to investigate https://github.com/toolforge/video2commons/issues/210 I started an ffmpeg update which unfortunately did not go as I hoped... So right now the backend is broken, I need to reinstall Gentoo and ffmpeg from scratch, might take a few hours. Until it is completed I have shut down the frontend so that no new tasks can be created. vip (talk) 16:14, 19 August 2024 (UTC)

Gentoo has been reinstalled and ffmpeg updated. I have restarted the service but not tested anything yet. Can some people please tell me if it works? vip (talk) 01:32, 20 August 2024 (UTC)

pika says yes. :D RZuo (talk) 06:01, 20 August 2024 (UTC)

An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/baeccb7c9e7e99ba/dl.mkv -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -speed 4 -f webm -ss 0 -an -pass 1 -passlogfile /srv/v2c/output/baeccb7c9e7e99ba/dl.mkv.vp9.webm.log /dev/null\\nExitcode: 8')"

while trying https://www.youtube.com/watch?v=5LW4T92FXYE . RZuo (talk) 06:40, 20 August 2024 (UTC)

OK, ffmpeg is not happy with one of the options we set: Unrecognized option 'row-mt'. vip (talk) 08:20, 20 August 2024 (UTC)

An exception occurred: UnpickleableExceptionWrapper: b"TaskError('/mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/416020d42922edea/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -crf 20 -qmin 1 -qmax 51 -b:v 0 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -speed 4 -f webm -ss 0 -an -pass 1 -passlogfile /srv/v2c/output/416020d42922edea/dl.mp4.vp9.webm.log /dev/null\\nExitcode: 8')"

with https://moraba.de/wp-content/uploads/2020/11/MAPHEUS5-2015-08-13_side-by-side_HD.mp4 Trasheater Midir (talk) 09:28, 20 August 2024 (UTC)

Thank you, it's the same error, I didn't install ffmpeg properly, all features are disabled. I am reinstalling it with required features. vip (talk) 09:32, 20 August 2024 (UTC)

I think it should work now, can someone please try again? vip (talk) 13:34, 20 August 2024 (UTC)

Launch confirmed. Trasheater Midir (talk) 16:18, 20 August 2024 (UTC)

Awesome video, thank you! vip (talk) 19:02, 20 August 2024 (UTC)

OK, started again. Your task is pending... Yann (talk) 17:26, 20 August 2024 (UTC)

Start a discussion about Commons:Video2commons

Talk pages are where people discuss how to make content on Wikimedia Commons the best that it can be. You can use this page to start a discussion with others about how to improve Commons:Video2commons.

Start a discussion