Commit Graph

277 Commits

Author SHA1 Message Date
pukkandan
81b4712bca
[extractor] Fix --load-pages 2023-07-22 09:00:44 +05:30
coletdjnz
3d2623a898
[compat, networking] Deprecate old functions (#2861)
Authored by: coletdjnz, pukkandan
2023-07-15 16:18:35 +05:30
coletdjnz
227bf1a33b
[networking] Rewrite architecture (#2861)
New networking interface consists of a `RequestDirector` that directs
each `Request` to appropriate `RequestHandler` and returns the
`Response` or raises `RequestError`. The handlers define adapters to
transform its internal Request/Response/Errors to our interfaces.

User-facing changes:
- Fix issues with per request proxies on redirects for urllib
- Support for `ALL_PROXY` environment variable for proxy setting
- Support for `socks5h` proxy
   - Closes https://github.com/yt-dlp/yt-dlp/issues/6325, https://github.com/ytdl-org/youtube-dl/issues/22618, https://github.com/ytdl-org/youtube-dl/pull/28093
- Raise error when using `https` proxy instead of silently converting it to `http`

Authored by: coletdjnz
2023-07-15 16:18:35 +05:30
pukkandan
bc344cd456
[core] Allow extractors to mark formats as potentially DRM (#7396)
This is useful for HLS where detecting whether the format is
actually DRM requires the child manifest to be downloaded.

Makes the error message when using `--test` inconsistent,
but doesn't really matter.
2023-07-06 21:40:01 +05:30
pukkandan
5fd8367496
[extractor] Support multiple _VALID_URLs (#5812)
Authored by: nixxo
2023-06-22 03:19:55 +05:30
pukkandan
d7cd97e8d8
Fix bug in db3ad8a676
Closes #7367
2023-06-21 12:13:27 +05:30
pukkandan
ad54c9130e
[cleanup] Misc
Closes #6288, Closes #7197, Closes #7265, Closes #7353, Closes #5773
Authored by: mikf, freezboltz, pukkandan
2023-06-21 09:21:20 +05:30
Nicolai Dagestad
db3ad8a676
Add option --netrc-cmd (#6682)
Authored by: NDagestad, pukkandan
Closes #1706
2023-06-21 08:37:42 +05:30
coletdjnz
8213ce28a4
[extractor/youtube] Extract channel_is_verified (#7213)
Authored by: coletdjnz
2023-06-08 19:15:39 +05:30
coletdjnz
c35448b7b1
[extractor/youtube] Extract more metadata for comments (#7179)
Adds new comment fields:
* `author_url` - The url to the comment author's page
* `author_is_verified` - Whether the author is verified on the platform
* `is_pinned` - Whether the comment is pinned to the top of the comments

Closes https://github.com/yt-dlp/yt-dlp/issues/5411

Authored by: coletdjnz
2023-06-01 08:43:32 +00:00
coletdjnz
b87e01c123
[cookies] Move YoutubeDLCookieJar to cookies module (#7091)
Authored by: coletdjnz
2023-05-27 07:08:19 +00:00
Audrey
5caf30dbc3
[extractor/youtube] Extract heatmap data (#7100)
Closes #3888
Authored by: tntmod54321
2023-05-26 17:54:39 +05:30
pukkandan
84ffeb7d5e
[extractor] Do not warn for invalid chapter data in description
Fixes https://github.com/yt-dlp/yt-dlp/issues/6811#issuecomment-1509876209
2023-04-16 08:55:43 +05:30
pukkandan
f68434cc74
[extractor] Extract more metadata from ISM
Fixes 81b6102d20 (r105892531)
2023-03-25 13:18:21 +05:30
pukkandan
baa922b5c7
[extractor] Do not exit early for unsuitable url_result 2023-03-25 13:18:21 +05:30
mushbite
22ccd5420b
[extractor/rutube] Extract chapters from description (#6345)
Authored by: mushbite
2023-03-04 19:03:17 +05:30
pukkandan
5b28cef72d
[cleanup] Misc 2023-02-28 23:51:06 +05:30
pukkandan
f34804b2f9
[extractor/youtube] Fix 5038f6d713
* [fragment] Fix `request_data`
* [youtube] Don't use POST for now. It may be easier to break in future

Authored by: bashonly, coletdjnz
2023-02-28 23:34:43 +05:30
pukkandan
43a3eaf963
[extractor] Fix DRM detection in m3u8
Fixes https://github.com/ytdl-org/youtube-dl/issues/31693#issuecomment-1445202857
2023-02-26 10:27:46 +05:30
pukkandan
45b2ee6f4f
Update to ytdl-commit-2dd6c6e
[YouTube] Avoid crash if uploader_id extraction fails
2dd6c6edd8

Except:
    * 295736c9cba714fb5de7d1c3dd31d86e50091cf8 [jsinterp] Improve parsing
    * 384f632e8a9b61e864a26678d85b2b39933b9bae [ITV] Overhaul ITV extractor
    * 33db85c571304bbd6863e3407ad8d08764c9e53b [feat]: Add support to external downloader aria2p
2023-02-17 17:52:23 +05:30
LowSuggestion912
b23167e754
[extractor/common] Fix _search_nuxt_data (#6062)
Authored by: LowSuggestion912
2023-02-12 12:55:24 +05:30
bashonly
7e68567e50
[downloader/hls] Allow extractors to provide AES key (#6158)
and related cleanup

Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
2023-02-08 11:09:32 +05:30
bashonly
5ab3534d44
[extractor/slideslive] Fix slides and chapters/duration (#6024)
* Fix slides/thumbnails extraction
* Extract duration to fix issues w/ `--embed-chapters`, `--split-chapters`
* Add `InfoExtractor._extract_mpd_vod_duration` method
* Expand applicability of `InfoExtractor._parse_m3u8_vod_duration` method
Authored by: bashonly
2023-01-14 19:52:03 +00:00
bashonly
cb73b8460c
[extractor/nbc] Fix NBC and NBCStations extractors (#6033)
Improve `InfoExtractor._parse_smil_formats` extension detection
Closes #6019
Authored by: bashonly
2023-01-14 16:40:42 +00:00
pukkandan
edfc7725b1
[cleanup] Misc 2023-01-07 02:48:34 +05:30
pukkandan
08e29b9f1f
[cleanup] Misc
Closes #5576, closes #5887
2023-01-02 19:40:15 +05:30
Simon Sawicki
32a84bcf4e
Update to ytdl-commit-195f22f6
[generic] Improve KVS (etc) extraction
195f22f679

Closes #3716
Authored by: Grub4k, pukkandan
2023-01-02 19:15:36 +05:30
Matthew
e756f45ba0
Improve handling for overriding extractors with plugins (#5916)
* Extractors replaced with plugin extractors now show in debug output
* Better testcase handling
* Added documentation
Authored by: coletdjnz, pukkandan
2023-01-02 04:55:11 +00:00
pukkandan
0b5546c723
[extractor] Let _extract_format functions obey --ignore-no-formats 2022-12-23 01:44:18 +05:30
pukkandan
71df9b7fd5
[cleanup] Misc 2022-12-03 19:52:31 +05:30
pukkandan
9bcfe33be7
[utils] Make ExtractorError mutable 2022-11-30 06:10:26 +05:30
pukkandan
9f14daf22b
[extractor] Deprecate _sort_formats 2022-11-17 11:40:17 +05:30
pukkandan
784320c98c
Implement universal format sorting
Closes #5566
2022-11-17 11:05:49 +05:30
pukkandan
d0d74b7197
[utils] Move format sorting code into utils 2022-11-17 11:04:38 +05:30
pukkandan
105bfd90f5
Add new field aspect_ratio
Closes #5402
2022-11-16 06:57:09 +05:30
pukkandan
6368e2e639
[cleanup] Misc
Closes #5541
2022-11-16 06:57:07 +05:30
pukkandan
171a31dbe8
[extractor] Add a way to distinguish IEs that returns only videos 2022-11-13 10:56:04 +05:30
pukkandan
f7fc8d39e9
[extractor] Fix fatal=False for _search_nuxt_data
Closes #5423
2022-11-11 07:29:29 +05:30
Matthew
0cf643b234
[extractor/youtube] Differentiate between no and disabled comments (#5491)
`comments` and `comment_count` will be set to None, as opposed to 
an empty list and 0, respectively.

Fixes https://github.com/yt-dlp/yt-dlp/issues/5068

Authored by: coletdjnz, pukkandan
2022-11-10 03:33:03 +00:00
pukkandan
46d09f8707
[cleanup] Lint and misc cleanup 2022-11-07 02:32:36 +05:30
pukkandan
62b8dac490
[extractor] Improve _generic_title 2022-10-31 17:41:48 +05:30
m4tu4g
c13a301a94
[extractor/zeenews] Add extractor (#5289)
Closes #4967 
Authored by: m4tu4g, pukkandan
2022-10-20 03:17:18 +05:30
pukkandan
d5d1df8afd
[cleanup Misc
Closes #5162
2022-10-18 23:52:44 +05:30
pukkandan
5225df50cf
[extractor/youtube:tab] Let approximate_date return timestamp 2022-10-13 15:30:15 +05:30
pukkandan
ade1fa70cb
[extractor/generic] Separate embed extraction into own function (#5176) 2022-10-09 16:09:36 +05:30
Simon Sawicki
1d55ebabc9
[extractor/common] Fix json_ld type checks (#5145)
Closes #5144, #5143
Authored by: Grub4K
2022-10-09 08:47:58 +05:30
Matthew
867c66ff97
[extractor/youtube] Extract concurrent view count for livestreams (#5152)
Adds new field `concurrent_view_count`
Closes https://github.com/yt-dlp/yt-dlp/issues/4843

Authored by: coletdjnz
2022-10-07 07:00:40 +00:00
pukkandan
a057779d5e
[cleanup] Minor fixes
Closes #5129, Closes #4982
2022-10-04 01:48:14 +05:30
coletdjnz
d3a3d7f0cc
[extractor/JWPlatform] Fix extractor (#5112)
Fix bitrate and filesize extraction and support embeds with unquoted urls.

Related: #5106 

Authored by: coletdjnz
2022-10-03 19:37:48 +00:00
pukkandan
8b7fb8b60d
[extractor] Make search_json able to parse lists
Now `contains_pattern` can be set to `\[.+\]`
2022-10-03 16:56:19 +05:30