mirror of
https://github.com/tcsenpai/TransTerm.git
synced 2025-06-02 17:30:20 +00:00
Initial commit
This commit is contained in:
commit
05c4393ba1
7
.gitignore
vendored
Normal file
7
.gitignore
vendored
Normal file
@ -0,0 +1,7 @@
|
||||
__pycache__
|
||||
*.mp3
|
||||
*.mp4
|
||||
*.wav
|
||||
downloads/*.mp3
|
||||
downloads/*.mp4
|
||||
downloads/*.wav
|
8
.trunk/.gitignore
vendored
Normal file
8
.trunk/.gitignore
vendored
Normal file
@ -0,0 +1,8 @@
|
||||
*out
|
||||
*logs
|
||||
*actions
|
||||
*notifications
|
||||
*tools
|
||||
plugins
|
||||
user_trunk.yaml
|
||||
user.yaml
|
2
.trunk/configs/.isort.cfg
Normal file
2
.trunk/configs/.isort.cfg
Normal file
@ -0,0 +1,2 @@
|
||||
[settings]
|
||||
profile=black
|
10
.trunk/configs/.markdownlint.yaml
Normal file
10
.trunk/configs/.markdownlint.yaml
Normal file
@ -0,0 +1,10 @@
|
||||
# Autoformatter friendly markdownlint config (all formatting rules disabled)
|
||||
default: true
|
||||
blank_lines: false
|
||||
bullet: false
|
||||
html: false
|
||||
indentation: false
|
||||
line_length: false
|
||||
spaces: false
|
||||
url: false
|
||||
whitespace: false
|
7
.trunk/configs/.shellcheckrc
Normal file
7
.trunk/configs/.shellcheckrc
Normal file
@ -0,0 +1,7 @@
|
||||
enable=all
|
||||
source-path=SCRIPTDIR
|
||||
disable=SC2154
|
||||
|
||||
# If you're having issues with shellcheck following source, disable the errors via:
|
||||
# disable=SC1090
|
||||
# disable=SC1091
|
10
.trunk/configs/.yamllint.yaml
Normal file
10
.trunk/configs/.yamllint.yaml
Normal file
@ -0,0 +1,10 @@
|
||||
rules:
|
||||
quoted-strings:
|
||||
required: only-when-needed
|
||||
extra-allowed: ["{|}"]
|
||||
empty-values:
|
||||
forbid-in-block-mappings: true
|
||||
forbid-in-flow-mappings: true
|
||||
key-duplicates: {}
|
||||
octal-values:
|
||||
forbid-implicit-octal: true
|
5
.trunk/configs/ruff.toml
Normal file
5
.trunk/configs/ruff.toml
Normal file
@ -0,0 +1,5 @@
|
||||
# Generic, formatter-friendly config.
|
||||
select = ["B", "D3", "E", "F"]
|
||||
|
||||
# Never enforce `E501` (line length violations). This should be handled by formatters.
|
||||
ignore = ["E501"]
|
43
.trunk/trunk.yaml
Normal file
43
.trunk/trunk.yaml
Normal file
@ -0,0 +1,43 @@
|
||||
# This file controls the behavior of Trunk: https://docs.trunk.io/cli
|
||||
# To learn more about the format of this file, see https://docs.trunk.io/reference/trunk-yaml
|
||||
version: 0.1
|
||||
cli:
|
||||
version: 1.18.1
|
||||
# Trunk provides extensibility via plugins. (https://docs.trunk.io/plugins)
|
||||
plugins:
|
||||
sources:
|
||||
- id: trunk
|
||||
ref: v1.4.1
|
||||
uri: https://github.com/trunk-io/plugins
|
||||
# Many linters and tools depend on runtimes - configure them here. (https://docs.trunk.io/runtimes)
|
||||
runtimes:
|
||||
enabled:
|
||||
- go@1.21.0
|
||||
- node@18.12.1
|
||||
- python@3.10.8
|
||||
# This is the section where you manage your linters. (https://docs.trunk.io/check/configuration)
|
||||
lint:
|
||||
enabled:
|
||||
- checkov@3.1.42
|
||||
- taplo@0.8.1
|
||||
- trivy@0.48.1
|
||||
- yamllint@1.33.0
|
||||
- bandit@1.7.6
|
||||
- black@23.12.0
|
||||
- dotenv-linter@3.3.0
|
||||
- git-diff-check
|
||||
- isort@5.13.2
|
||||
- markdownlint@0.38.0
|
||||
- osv-scanner@1.5.0
|
||||
- prettier@3.1.1
|
||||
- ruff@0.1.9
|
||||
- shellcheck@0.9.0
|
||||
- shfmt@3.6.0
|
||||
- trufflehog@3.63.5
|
||||
actions:
|
||||
disabled:
|
||||
- trunk-check-pre-push
|
||||
- trunk-fmt-pre-commit
|
||||
enabled:
|
||||
- trunk-announce
|
||||
- trunk-upgrade-available
|
7
LICENSE.md
Normal file
7
LICENSE.md
Normal file
@ -0,0 +1,7 @@
|
||||
Copyright 2023, TheCookingSenpai
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
123
README.md
Normal file
123
README.md
Normal file
@ -0,0 +1,123 @@
|
||||
# TransTerm
|
||||
|
||||
A terminal based YouTube video and playlist downloader (mp4, mp3, wav) and transcriber with a nice TUI interface.
|
||||
|
||||

|
||||
|
||||
PS. The transparency and retro term is proper of my KDE Plasma configuration
|
||||
|
||||
🌲🌲🌲🦌🌲🌲🌲🦌🌲🌲🌲
|
||||
|
||||
**By utilizing TransTerm, you acknowledge that you have read, comprehended, and agreed to the terms set forth in the disclaimer at the bottom of this file.**
|
||||
|
||||
# Table of Contents
|
||||
|
||||
1. [What is TransTerm?](#what-is-this)
|
||||
2. [Features](#features)
|
||||
3. [Installation](#installation)
|
||||
- [For Mac & Linux](#installation-for-mac--linux)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
- [For Windows](#installation-for-windows)
|
||||
4. [Usage](#usage)
|
||||
5. [Manual Mode (Debug)](#manual-mode-aka-debug)
|
||||
6. [Contributing](#contributing)
|
||||
7. [License](#license)
|
||||
8. [Disclaimer](#disclaimer)
|
||||
- [Terms and Conditions](#terms-and-conditions)
|
||||
- [Appropriate Use](#appropriate-use)
|
||||
- [Copyright Adherence](#copyright-adherence)
|
||||
- [Personal Usage](#personal-usage)
|
||||
- [Third-Party Content](#third-party-content)
|
||||
- [Limitation of Responsibility](#limitation-of-responsibility)
|
||||
|
||||
## What is this
|
||||
|
||||
TransTerm is an highly experimental text based graphical user interface to act on YouTube videos.
|
||||
|
||||
Being text based, this program runs even in the terminal.
|
||||
|
||||
## Features
|
||||
|
||||
- Download any youtube video at the highest resolution by default in mp4 format
|
||||
- Is able to automatically convert the downloaded video both in mp3 or wav format
|
||||
- Playlist support for the above features including automatically rename the files using the video title and the channel name
|
||||
- Transcribe a ssingle downloaded video using either:
|
||||
• Google Audio to Text
|
||||
• Google Audio to Text + Silence detection
|
||||
• Sphynx CMU (processed offline locally)
|
||||
|
||||
## Installation for Mac & Linux
|
||||
|
||||
git clone https://github.com/thecookingsenpai/transterm && cd transterm
|
||||
|
||||
sudo chmod +x install.sh
|
||||
|
||||
./install.sh
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
In some cases, where python3 environment is managed by the system and not by pip3, it is advisable to install the dependencies manually as you would do with your system. If you want to force the installation anyway, first run:
|
||||
|
||||
pip3 install -r requirements.txt --break-system-packages
|
||||
|
||||
The same general solution applies to any requirements problem. PLease note that using the above command is not recommended as it could break your python installation.
|
||||
|
||||
## Installation for Window
|
||||
|
||||
git clone https://github.com/thecookingsenpai/transterm && cd transterm && pip3 install -r requirements.txt
|
||||
|
||||
## Usage
|
||||
|
||||
python3 gui.py
|
||||
|
||||
## Manual mode (aka debug)
|
||||
|
||||
You can edit the bottom part of term.py so that after the
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
condition you can write your own logic. Then, launch with:
|
||||
|
||||
python3 term.py
|
||||
|
||||
For convenience, a playlist example is already included (with the best playlist ever, I'd say).
|
||||
|
||||
## Contributing
|
||||
|
||||
You are welcome and free to contribute to the project. To do that, you have a few ways:
|
||||
|
||||
- Add some kind of functionality and create a PR
|
||||
- Search the lines marked with TODO and FIXME in the code for the most urgent things
|
||||
- In any case, thank you!
|
||||
|
||||
## License
|
||||
|
||||
MIT License
|
||||
|
||||
## Disclaimer
|
||||
|
||||
### Terms and Conditions:
|
||||
|
||||
TransTerm is a YouTube downloader and transcriber application developed for personal, non-commercial use. Users must ensure that their utilization of this tool complies with all relevant laws and regulations in their jurisdiction.
|
||||
|
||||
### Appropriate Use:
|
||||
|
||||
#### Copyright Adherence:
|
||||
|
||||
Users must respect the copyrights and intellectual property rights of content creators. TransTerm should not be utilized to infringe upon copyrights or distribute content without proper authorization.
|
||||
|
||||
#### Personal Usage:
|
||||
|
||||
The application is intended for personal use, allowing users to download and transcribe YouTube videos for lawful purposes such as education, research, or similar activities. Commercial use or distribution of downloaded/transcribed content without authorization is strictly prohibited.
|
||||
|
||||
#### Third-Party Content:
|
||||
|
||||
Users are responsible for confirming that they possess the legal rights to download and transcribe any third-party content from YouTube. TransTerm does not endorse or encourage the unauthorized downloading or distribution of copyrighted materials.
|
||||
|
||||
### Limitation of Responsibility:
|
||||
|
||||
The developers of TransTerm are not accountable for any misuse or illegal use of the tool. Users accept all associated risks with their use of the application and agree to comply with applicable laws and YouTube's terms of service.
|
||||
|
||||
**_Please Note: This disclaimer does not serve as legal advice. Users are encouraged to seek legal counsel if they have queries regarding the legality of their use of TransTerm in their specific jurisdiction._**
|
||||
|
||||
**By utilizing TransTerm, you acknowledge that you have read, comprehended, and agreed to the terms set forth in this disclaimer.**
|
1
downloads/placeholder
Normal file
1
downloads/placeholder
Normal file
@ -0,0 +1 @@
|
||||
why are you looking at me, my only purpose is to create my parent folder
|
238
gui.py
Normal file
238
gui.py
Normal file
@ -0,0 +1,238 @@
|
||||
import random
|
||||
import threading
|
||||
from statistics import mean
|
||||
|
||||
from textual import events
|
||||
from textual.app import App, ComposeResult
|
||||
from textual.containers import Horizontal
|
||||
from textual.widgets import (
|
||||
Button,
|
||||
Checkbox,
|
||||
Footer,
|
||||
Header,
|
||||
Input,
|
||||
Label,
|
||||
RichLog,
|
||||
Select,
|
||||
Sparkline,
|
||||
)
|
||||
|
||||
import term
|
||||
|
||||
|
||||
class TransTerm(App):
|
||||
CSS_PATH = "meshterm.tcss"
|
||||
|
||||
lock = False
|
||||
|
||||
stopWatchdog = False
|
||||
messageToShow = None
|
||||
|
||||
# INFO Composing the app
|
||||
def compose(self) -> ComposeResult:
|
||||
"""Create child widgets for the app."""
|
||||
yield Header()
|
||||
yield Footer()
|
||||
yield Label(
|
||||
"TransTerm - A simple terminal-based YouTube downloader and transcriber by TheCookingSenpai\n",
|
||||
classes="title",
|
||||
)
|
||||
yield Label(
|
||||
"Yellow -> idle • Red -> busy • Green -> success\n\n",
|
||||
classes="status_yellow",
|
||||
id="status",
|
||||
)
|
||||
# Inputs
|
||||
|
||||
yield Label("Enter a YouTube link to work with:")
|
||||
yield Input(
|
||||
placeholder="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
|
||||
id="link",
|
||||
)
|
||||
# Configuration
|
||||
yield Horizontal(
|
||||
Checkbox(
|
||||
"Download a playlist",
|
||||
id="downloadPlaylist",
|
||||
),
|
||||
Checkbox(
|
||||
"Convert to mp3",
|
||||
id="toMp3",
|
||||
),
|
||||
Checkbox(
|
||||
"Convert to wav",
|
||||
id="toWav",
|
||||
),
|
||||
Checkbox(
|
||||
"Transcript to text",
|
||||
id="toText",
|
||||
),
|
||||
)
|
||||
options = [
|
||||
("Google Simple", "google"),
|
||||
("Google with silence detection", "google_silence"),
|
||||
("Local (using Sphynx)", "local"),
|
||||
]
|
||||
yield Select(options, id="engine")
|
||||
# Buttons
|
||||
yield Button("Go", id="go")
|
||||
yield Button("Exit", id="exit")
|
||||
# Infos
|
||||
yield Label("Video title: ", id="video_title")
|
||||
yield Label("Video author: ", id="video_author")
|
||||
yield Label("Video length: ", id="video_length")
|
||||
yield Label("Configuration: ", id="configuration")
|
||||
random.seed(73)
|
||||
data = [random.expovariate(1 / 3) for _ in range(1000)]
|
||||
yield Sparkline(data, summary_function=mean, id="divisor")
|
||||
yield RichLog(id="main_log")
|
||||
|
||||
# SECTION Actions
|
||||
def on_key(self, event: events.Key) -> None:
|
||||
"""Handle key events."""
|
||||
pass
|
||||
|
||||
def on_button_pressed(self, event: Button.Pressed) -> None:
|
||||
# sourcery skip: extract-method, switch
|
||||
"""Handle button events."""
|
||||
action = str(event.button.id).lower()
|
||||
if action == "exit":
|
||||
try:
|
||||
term.forceQuit = True
|
||||
except Exception:
|
||||
print("[SYSTEM] Failed to stop thread")
|
||||
exit(1)
|
||||
elif action == "go":
|
||||
if self.lock:
|
||||
self.query_one("#main_log").write("Already working!")
|
||||
return
|
||||
self.lock = True
|
||||
status = self.query_one("#status")
|
||||
status.classes = "status_red"
|
||||
status.text = "Status: working..."
|
||||
self.query_one("#main_log").write("Proceeding...")
|
||||
self.query_one("#main_log").write("Params extraction...")
|
||||
self.process = threading.Thread(name="act", target=self.act)
|
||||
self.query_one("#main_log").write("Started!")
|
||||
self.process.start()
|
||||
|
||||
def act(self):
|
||||
link = self.query_one("#link")
|
||||
link = link.value
|
||||
if link == "":
|
||||
link = "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
|
||||
self.query_one("#main_log").write(f"Link: {link}")
|
||||
toMp3 = self.query_one("#toMp3")
|
||||
toMp3 = toMp3.value
|
||||
toWav = self.query_one("#toWav")
|
||||
toWav = toWav.value
|
||||
toText = self.query_one("#toText")
|
||||
toText = toText.value
|
||||
isPlaylist = self.query_one("#downloadPlaylist")
|
||||
isPlaylist = isPlaylist.value
|
||||
|
||||
# Branching: in playlist mode we download (and if needed convert) all the videos of the playlist
|
||||
# TODO NOTE: Transcribing is intentionally disabled on playlists. Feel free to tinker but it's a risk.
|
||||
|
||||
# TODO: Avoid redundancy with the below branch
|
||||
if isPlaylist:
|
||||
self.query_one("#main_log").write("Downloading playlist...")
|
||||
if toMp3 or toWav:
|
||||
self.query_one("#main_log").write(
|
||||
"Converting to mp3 only too (wav is not supported on playlists)...this may take a while"
|
||||
)
|
||||
playlist = term.printPlaylist(link)
|
||||
print(playlist)
|
||||
self.query_one("#main_log").write(
|
||||
"Please note that the TUI might seems frozen while downloading the playlist. Check the downloads folder for the progress."
|
||||
)
|
||||
self.query_one("#main_log").write("...Yes, we are working on it.")
|
||||
# TODO More verbosity!
|
||||
d_path = term.managePlaylist(
|
||||
playlist, to_download=True, to_convert=toMp3 or toWav
|
||||
)
|
||||
status = self.query_one("#status")
|
||||
status.classes = "status_green"
|
||||
self.query_one("#main_log").write("Done!")
|
||||
self.query_one("#main_log").write(str(d_path))
|
||||
self.lock = False
|
||||
return
|
||||
|
||||
# Branching: in single video mode we download (and if needed convert) the video
|
||||
self.query_one("#main_log").write(f"MP3: {toMp3}, WAV: {toWav}, TEXT: {toText}")
|
||||
try:
|
||||
infos = term.getInfo(link)
|
||||
except Exception as e:
|
||||
self.query_one("#main_log").write(
|
||||
"ERROR: Could not retrieve informations: " + str(e) + ";"
|
||||
)
|
||||
return
|
||||
|
||||
self.query_one("#video_title").value = "Title: " + infos["title"]
|
||||
self.query_one("#video_author").value = "Author: " + infos["author"]
|
||||
self.query_one("#video_length").value = "Seconds: " + infos["length"]
|
||||
self.query_one("#configuration").value = (
|
||||
"\n" + f"MP3: {toMp3}, WAV: {toWav}, TEXT: {toText}"
|
||||
)
|
||||
|
||||
# First we download the video
|
||||
folder = term.download(link)
|
||||
# Now, if the user wants to convert to mp3, we do it
|
||||
if toMp3:
|
||||
self.query_one("#main_log").write(
|
||||
"Converting to mp3...this may take a while"
|
||||
)
|
||||
term.convert(folder, format="mp3")
|
||||
# Now, if the user wants to convert to wav or text, we do it
|
||||
if toWav and not toText:
|
||||
self.query_one("#main_log").write(
|
||||
"Converting to wav...this may take a while"
|
||||
)
|
||||
file = term.convert(folder, format="wav")
|
||||
# Now, if the user wants to convert to text, we do it
|
||||
if toText:
|
||||
self.query_one("#main_log").write(
|
||||
"Converting to text...this may take a while"
|
||||
)
|
||||
engine = self.query_one("#engine").value
|
||||
file = term.convert(folder, format="wav")
|
||||
if engine == "google_silence":
|
||||
self.query_one("#main_log").write("Using Google with silence detection")
|
||||
file = term.get_large_audio_transcription_on_silence(folder)
|
||||
elif engine == "local":
|
||||
self.query_one("#main_log").write("Using Sphynx CMU")
|
||||
file = term.local_audio_transcribe(folder)
|
||||
else:
|
||||
self.query_one("#main_log").write("Using Google Simple")
|
||||
file = term.simple_audio_transcribe(folder)
|
||||
status = self.query_one("#status")
|
||||
status.classes = "status_green"
|
||||
self.query_one("#main_log").write("Done!")
|
||||
self.query_one("#main_log").write(str(file))
|
||||
self.lock = False
|
||||
return True
|
||||
|
||||
# !SECTION Actions
|
||||
|
||||
def loadEnv(self):
|
||||
self.env = {}
|
||||
with open(".env", "r") as f:
|
||||
textenv = f.readlines()
|
||||
for line in textenv:
|
||||
key, value = line.split("=")
|
||||
self.env[key.strip()] = value.strip()
|
||||
|
||||
def saveEnv(self): # sourcery skip: use-join
|
||||
preparedEnv = ""
|
||||
for key, value in self.env.items():
|
||||
preparedEnv += f"{key}={value}" + "\n"
|
||||
with open(".env", "w") as f:
|
||||
f.write(preparedEnv)
|
||||
f.flush()
|
||||
return self.env
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
app = TransTerm()
|
||||
app.loadEnv()
|
||||
app.run()
|
32
install.sh
Normal file
32
install.sh
Normal file
@ -0,0 +1,32 @@
|
||||
#!/bin/bash
|
||||
|
||||
pip3 install -r requirements.txt
|
||||
#pip3 install -r requirements.txt --break-system-packages
|
||||
|
||||
echo "Preparing to install PortAudio for Python"
|
||||
|
||||
unameOut="$(uname -s)"
|
||||
case "${unameOut}" in
|
||||
Linux*) machine=Linux ;;
|
||||
Darwin*) machine=Mac ;;
|
||||
CYGWIN*) machine=Cygwin ;;
|
||||
MINGW*) machine=MinGw ;;
|
||||
*) machine="UNKNOWN:${unameOut}" ;;
|
||||
esac
|
||||
|
||||
if [[ ${machine} == "Mac" ]]; then
|
||||
# check if homebrew is installed on Macos if not install Homebrew
|
||||
echo "Checking Homebrew"
|
||||
command -v brew >/dev/null 2>&1 || {
|
||||
echo >&2 "Installing Homebrew Now"
|
||||
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
|
||||
}
|
||||
# if homebrew is installed, update to latest
|
||||
brew update-reset
|
||||
# install portaudio
|
||||
brew install portaudio
|
||||
pip3 install pyaudio
|
||||
elif [[ "$(expr substr $(uname -s) 1 5)" == "Linux" ]]; then
|
||||
sudo apt-get install -y portaudio19-dev
|
||||
sudo apt-get install python3-pyaudio
|
||||
fi
|
39
meshterm.tcss
Normal file
39
meshterm.tcss
Normal file
@ -0,0 +1,39 @@
|
||||
Button {
|
||||
margin: 1 2;
|
||||
}
|
||||
|
||||
|
||||
|
||||
.header {
|
||||
margin: 1 0 0 2;
|
||||
text-style: bold;
|
||||
}
|
||||
|
||||
Sparkline {
|
||||
width: 100%;
|
||||
margin: 2;
|
||||
color: red;
|
||||
}
|
||||
|
||||
#divisor > .sparkline--max-color {
|
||||
color: orange;
|
||||
}
|
||||
#divisor > .sparkline--min-color {
|
||||
color: red;
|
||||
}
|
||||
|
||||
.title {
|
||||
color: red;
|
||||
}
|
||||
|
||||
.status_yellow {
|
||||
color: yellow
|
||||
}
|
||||
|
||||
.status_red {
|
||||
color: red
|
||||
}
|
||||
|
||||
.status_green {
|
||||
color: green
|
||||
}
|
8
requirements.txt
Normal file
8
requirements.txt
Normal file
@ -0,0 +1,8 @@
|
||||
setuptools
|
||||
wheel
|
||||
pydub
|
||||
pyaudio
|
||||
pocketsphinx
|
||||
SpeechRecognition
|
||||
dotenv
|
||||
textual
|
208
term.py
Normal file
208
term.py
Normal file
@ -0,0 +1,208 @@
|
||||
# sourcery skip: use-fstring-for-concatenation
|
||||
import contextlib
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import unicodedata
|
||||
|
||||
import speech_recognition as sr
|
||||
from pydub import AudioSegment
|
||||
from pydub.silence import split_on_silence
|
||||
from pytube import Playlist, YouTube
|
||||
|
||||
forceQuit = False
|
||||
r = sr.Recognizer()
|
||||
|
||||
|
||||
# NOTE: Taken from https://stackoverflow.com/questions/295135/turn-a-string-into-a-valid-filename
|
||||
def slugify(value, allow_unicode=False):
|
||||
"""
|
||||
Taken from https://github.com/django/django/blob/master/django/utils/text.py
|
||||
Convert to ASCII if 'allow_unicode' is False. Convert spaces or repeated
|
||||
dashes to single dashes. Remove characters that aren't alphanumerics,
|
||||
underscores, or hyphens. Convert to lowercase. Also strip leading and
|
||||
trailing whitespace, dashes, and underscores.
|
||||
"""
|
||||
value = str(value)
|
||||
if allow_unicode:
|
||||
value = unicodedata.normalize("NFKC", value)
|
||||
else:
|
||||
value = (
|
||||
unicodedata.normalize("NFKD", value)
|
||||
.encode("ascii", "ignore")
|
||||
.decode("ascii")
|
||||
)
|
||||
value = re.sub(r"[^\w\s-]", "", value.lower())
|
||||
return re.sub(r"[-\s]+", "-", value).strip("-_")
|
||||
|
||||
|
||||
def local_audio_transcribe(path):
|
||||
with sr.AudioFile(path) as source:
|
||||
audio_listened = r.record(source)
|
||||
# try converting it to text
|
||||
r.recognize_sphinx(audio_listened, language="en-US", show_all=False)
|
||||
|
||||
|
||||
# a function to recognize speech in the audio file
|
||||
# so that we don't repeat ourselves in in other functions
|
||||
def simple_audio_transcribe(path):
|
||||
with sr.AudioFile(path) as source:
|
||||
# listen for the data (load audio to memory)
|
||||
audio_data = r.record(source)
|
||||
# recognize (convert from speech to text)
|
||||
text = r.recognize_google(audio_data)
|
||||
print(text)
|
||||
return text
|
||||
|
||||
|
||||
def transcribe_audio(path):
|
||||
# use the audio file as the audio source
|
||||
with sr.AudioFile(path) as source:
|
||||
audio_listened = r.record(source)
|
||||
# try converting it to text
|
||||
text = r.recognize_google(audio_listened)
|
||||
return text
|
||||
|
||||
|
||||
# TODO FIXME a function that splits the audio file into chunks on silence
|
||||
# and applies speech recognition (mostly unused, should be fixed and tested)
|
||||
def get_large_audio_transcription_on_silence(path):
|
||||
"""Splitting the large audio file into chunks
|
||||
and apply speech recognition on each of these chunks"""
|
||||
# open the audio file using pydub
|
||||
sound = AudioSegment.from_file(path)
|
||||
# split audio sound where silence is 500 miliseconds or more and get chunks
|
||||
chunks = split_on_silence(
|
||||
sound,
|
||||
# experiment with this value for your target audio file
|
||||
min_silence_len=1000,
|
||||
# adjust this per requirement
|
||||
silence_thresh=sound.dBFS - 14,
|
||||
# keep the silence for 1 second, adjustable as well
|
||||
keep_silence=500,
|
||||
)
|
||||
folder_name = "audio-chunks"
|
||||
# create a directory to store the audio chunks
|
||||
if not os.path.isdir(folder_name):
|
||||
os.mkdir(folder_name)
|
||||
whole_text = ""
|
||||
# process each chunk
|
||||
for i, audio_chunk in enumerate(chunks, start=1):
|
||||
# export audio chunk and save it in
|
||||
# the `folder_name` directory.
|
||||
chunk_filename = os.path.join(folder_name, f"chunk{i}.wav")
|
||||
audio_chunk.export(chunk_filename, format="wav")
|
||||
# recognize the chunk
|
||||
try:
|
||||
text = transcribe_audio(chunk_filename)
|
||||
except sr.UnknownValueError as e:
|
||||
print("Error:", e)
|
||||
else:
|
||||
text = f"{text.capitalize()}. "
|
||||
print(chunk_filename, ":", text)
|
||||
whole_text += text
|
||||
# return the text for all chunks detected
|
||||
return whole_text
|
||||
|
||||
|
||||
# INFO Cleanup the leftovers
|
||||
def cleanup():
|
||||
with contextlib.suppress(FileNotFoundError):
|
||||
os.remove("audio.wav")
|
||||
with contextlib.suppress(FileNotFoundError):
|
||||
os.remove("video.mp4")
|
||||
with contextlib.suppress(FileNotFoundError):
|
||||
os.remove("audio.mp3")
|
||||
shutil.rmtree("audio-chunks")
|
||||
|
||||
|
||||
# INFO Main youtube to mp4 downloader method
|
||||
def download(link, target_filename="video"):
|
||||
url = YouTube(link)
|
||||
print("downloading....")
|
||||
video = url.streams.get_highest_resolution()
|
||||
print(video.title)
|
||||
path_to_download_folder = (
|
||||
str(os.path.dirname(os.path.realpath(__file__))) + "/downloads"
|
||||
)
|
||||
video.download(path_to_download_folder, filename=target_filename + ".mp4")
|
||||
print("Downloaded! :)")
|
||||
return path_to_download_folder
|
||||
|
||||
|
||||
# INFO Print a playlist info
|
||||
def printPlaylist(playlist_link):
|
||||
playlist = Playlist(playlist_link)
|
||||
return playlist
|
||||
|
||||
|
||||
# INFO Download a playlist
|
||||
def managePlaylist(playlist, to_download=False, to_convert=False, named=True):
|
||||
print("Received playlist:", playlist)
|
||||
if not to_download:
|
||||
return playlist
|
||||
counter = 0
|
||||
path_to_download_folder = (
|
||||
str(os.path.dirname(os.path.realpath(__file__))) + "/downloads"
|
||||
)
|
||||
for url in playlist:
|
||||
counter += 1
|
||||
print("Downloading video", counter, "of", len(playlist))
|
||||
ytvideo = YouTube(url)
|
||||
video = ytvideo.streams.get_highest_resolution()
|
||||
filename = "video_" + str(counter)
|
||||
# Experimental name support
|
||||
if named:
|
||||
filename = slugify(ytvideo.title + "_" + ytvideo.author)
|
||||
print("Downloading : ", filename)
|
||||
video.download(path_to_download_folder, filename=filename + ".mp4")
|
||||
if to_convert:
|
||||
convert(
|
||||
path_to_download_folder,
|
||||
format="mp3",
|
||||
source_filename=filename,
|
||||
target_filename=filename,
|
||||
)
|
||||
print("Downloaded all videos! :)")
|
||||
return path_to_download_folder
|
||||
|
||||
|
||||
# INFO Get info about the video
|
||||
def getInfo(link):
|
||||
print("Getting info for", link)
|
||||
url = YouTube(link)
|
||||
# Trying to sanitize the title
|
||||
title = url.title + "_" + url.author
|
||||
title = slugify(title)
|
||||
return {
|
||||
"title": url.title,
|
||||
"author": url.author,
|
||||
"length": str(url.length),
|
||||
"filename": title,
|
||||
}
|
||||
|
||||
|
||||
# INFO Convert the video to audio
|
||||
def convert(
|
||||
path_to_download_folder,
|
||||
format="wav",
|
||||
source_filename="video",
|
||||
target_filename="audio",
|
||||
):
|
||||
src = f"{path_to_download_folder}/{source_filename}.mp4"
|
||||
dst = f"{path_to_download_folder}/{target_filename}.{format}"
|
||||
print("Converting to audio....")
|
||||
sound = AudioSegment.from_file(src, format="mp4")
|
||||
sound.export(dst, format=format)
|
||||
print("Converted to audio! :)")
|
||||
return dst
|
||||
|
||||
|
||||
# Testing (at the moment it is testing with a random playlist)
|
||||
if __name__ == "__main__":
|
||||
with contextlib.suppress(FileNotFoundError):
|
||||
cleanup()
|
||||
os.remove("transcription.txt")
|
||||
link = "https://www.youtube.com/playlist?list=PLXaw-xRbZ6E_LHJNJiIu-fOUo4SfpjHqE"
|
||||
playlist = printPlaylist(link)
|
||||
managePlaylist(playlist, to_download=True, to_convert=True)
|
Loading…
x
Reference in New Issue
Block a user