Compare commits

...

48 Commits

Author SHA1 Message Date
ilike2burnthing
3dd3e7559d
u_c: remove apparent c&p typo
https://github.com/ultrafunkamsterdam/undetected-chromedriver/pull/1933
2025-06-06 08:04:00 +01:00
ilike2burnthing
f21c1d51bc
Restore example service file. #1204 2025-06-04 23:13:00 +01:00
ilike2burnthing
957347f73a
Bump version 3.3.24 (#1505) 2025-06-04 19:02:06 +01:00
ilike2burnthing
c55080b0ec
Remove hidden character 2025-06-04 18:54:48 +01:00
ilike2burnthing
639bfca020
Bump version 3.3.23 (#1504) 2025-06-04 18:51:31 +01:00
ilike2burnthing
237694df76
Update base image to bookworm. resolves #1503 2025-06-04 18:44:17 +01:00
ilike2burnthing
6e5d6f1795
Bump version 3.3.22 (#1500) 2025-06-03 05:54:25 +01:00
ilike2burnthing
30804a86e5
Bump Chromium to v137 for build 2025-06-03 05:39:46 +01:00
ilike2burnthing
e0bdaf7745
Don't open devtools 2025-06-03 05:38:55 +01:00
ilike2burnthing
795365dbe4
Change from click to keys
credit to @sh4dowb - #1497
2025-06-03 05:38:06 +01:00
ilike2burnthing
ce5369dd41
Update bug_report.yml
fix accidental delete
2025-03-04 02:05:23 +00:00
ilike2burnthing
600b09d498
Update bug_report.yml 2025-03-04 02:04:29 +00:00
ilike2burnthing
d1f19405a1
Update README.md. closes #1267 2025-01-21 20:27:36 +00:00
ilike2burnthing
82a1366d34
Remove dead directory link from readme. resolve #1436 2025-01-20 17:37:14 +00:00
dependabot[bot]
a2fe9e7776
Bump waitress from 2.1.2 to 3.0.1 (#1418)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-29 01:53:34 +00:00
ilike2burnthing
6cc628df9e
Disable search engine choice screen 2024-11-24 18:31:50 +00:00
Eduard Tykhoniuk
8b1851eeb1
Fix headless=false stalling 2024-11-24 18:30:35 +00:00
ilike2burnthing
54668a11e7
bug_report: lint fix again 2024-09-28 08:08:49 +01:00
ilike2burnthing
701d8fb4ff
bug_report: change to input 2024-09-28 08:07:56 +01:00
ilike2burnthing
39a265ccb8
bug_report: lint fix 2024-09-28 07:58:14 +01:00
ilike2burnthing
e32b247014
bug_report: default=0 2024-09-28 07:57:11 +01:00
ilike2burnthing
0d8fe8fe50
bug_report: no booleans allowed 2024-09-28 07:51:57 +01:00
ilike2burnthing
718da3a36f
bug_report: add 'no really' drop down
maybe this will help... 🙄
2024-09-28 07:50:11 +01:00
ilike2burnthing
a798561338
Bump requests version
*.0 was yanked
2024-07-30 02:38:13 +01:00
Bogdan
eb680efc90
Don't build docker images for PRs from forks (#1281) 2024-07-20 22:08:40 +03:00
ilike2burnthing
0f8f0bec25
revert and bump action version 2024-07-20 19:41:49 +01:00
ilike2burnthing
3d9bc5627b
Change to GITHUB_TOKEN for GHRC login 2024-07-20 14:21:34 +01:00
ilike2burnthing
dd7eaee2e3
Bump requirements
resolves Dependabot alerts
2024-07-12 17:11:40 +01:00
ilike2burnthing
031177bbdb
Bump version 3.3.21 (#1240) 2024-06-26 02:14:25 +01:00
Bogdan
a8644532a1
Escape values for generated form used in request.post (#1236)
and build docker images for PRs
2024-06-26 02:04:59 +01:00
ilike2burnthing
e96161c873
Add challenge selector to catch reloading page on non-English systems. resolves #1237 2024-06-25 22:32:06 +01:00
ilike2burnthing
5a1f25cd52
Bump version 3.3.20 (#1229) 2024-06-21 22:21:37 +01:00
tenettow
a2c0e4348e
Update Cloudflare challenge and checkbox selectors (#1224) 2024-06-21 22:07:03 +01:00
ilike2burnthing
2ecf88895b
Check not running in Docker before logging version_main error 2024-06-15 08:37:42 +01:00
ilike2burnthing
984368edb5
maxTimeout should always be int. resolves #1212 2024-06-15 05:41:45 +01:00
21hsmw
6c1d78cb84
Fix occasional headless issue on Linux when set to "false" (#1199)
* Fix occasional headless issue on Linux when set to "false"

- Add a variable containing the current platform
- Check if the platform is "nt" (Windows) before closing the driver

* Update CHANGELOG.md

---------

Co-authored-by: ilike2burnthing <59480337+ilike2burnthing@users.noreply.github.com>
2024-05-24 17:33:46 +01:00
ilike2burnthing
5a2c61601e
Fix Chrome v124+ not closing on Windows. resolves #1161 (#1193) 2024-05-20 00:52:55 +01:00
ilike2burnthing
c304da2964
Update README.md 2024-04-22 23:30:52 +01:00
ilike2burnthing
b811412699
Fix LANG ENV for Linux. #1036 2024-04-20 03:41:53 +01:00
Ross Patterson
0bb8de144f
Add Compose V2 command to readme (#1154)
Co-authored-by: root@library.moodysalon.net <root@library.moodysalon.net>
Co-authored-by: ilike2burnthing <59480337+ilike2burnthing@users.noreply.github.com>
2024-04-18 05:07:06 +01:00
ilike2burnthing
38166dfaa0
Bump version 3.3.17 (#1147) 2024-04-09 20:31:21 +01:00
ilike2burnthing
8dea0ed017
Fix file descriptor leak in service on quit(). resolves #983
credit: @zkulis - https://github.com/ultrafunkamsterdam/undetected-chromedriver/pull/1812
2024-04-09 20:27:42 +01:00
francisco-lafe
20cd2944a7
Update README.md (#1127)
Co-authored-by: ilike2burnthing <59480337+ilike2burnthing@users.noreply.github.com>
2024-03-19 18:05:11 +00:00
ilike2burnthing
fd773e5909
Bump version 3.3.16 (#1105) 2024-02-28 21:38:53 +00:00
Justin Kromlinger
35c7bff3c8
Use headless configuration properly (#1104) 2024-02-28 20:36:38 +00:00
Raphaël
afdc1c7a8e
Add FreeBSD support (#1054) 2024-02-28 02:45:04 +00:00
ilike2burnthing
0bc7a4498c
Update README.md 2024-02-23 05:01:00 +00:00
Xewdy
c5a5f6d65e
Add platform specifiers to dependencies (#877) 2024-02-21 21:14:25 +00:00
16 changed files with 255 additions and 133 deletions

View File

@ -29,6 +29,13 @@ body:
options:
- label: I have read the Discussions
required: true
- type: input
attributes:
label: Have you ACTUALLY checked all these?
description: Please do not waste our time and yours; these checks are there for a reason, it is not just so you can tick boxes for fun. If you type <b>YES</b> and it is clear you did not or have put in no effort, your issue will be closed and locked without comment. If you type <b>NO</b> but still open this issue, you will be permanently blocked for timewasting.
placeholder: YES or NO
validations:
required: true
- type: textarea
attributes:
label: Environment

View File

@ -4,50 +4,64 @@ on:
push:
tags:
- 'v*.*.*'
pull_request:
branches:
- master
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
build-docker-images:
if: ${{ !github.event.pull_request.head.repo.fork }}
runs-on: ubuntu-22.04
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Downcase repo
- name: Checkout
uses: actions/checkout@v4
- name: Downcase repo
run: echo REPOSITORY=$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]') >> $GITHUB_ENV
-
name: Docker meta
- name: Docker meta
id: docker_meta
uses: crazy-max/ghaction-docker-meta@v3
uses: docker/metadata-action@v5
with:
images: ${{ env.REPOSITORY }},ghcr.io/${{ env.REPOSITORY }}
tag-sha: false
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Login to DockerHub
uses: docker/login-action@v2
images: |
${{ env.REPOSITORY }},enable=${{ github.event_name != 'pull_request' }}
ghcr.io/${{ env.REPOSITORY }}
tags: |
type=semver,pattern={{version}},prefix=v
type=ref,event=pr
flavor: |
latest=auto
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
uses: docker/login-action@v3
if: github.event_name != 'pull_request'
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Login to GitHub Container Registry
uses: docker/login-action@v2
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GH_PAT }}
-
name: Build and push
uses: docker/build-push-action@v3
- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: ./Dockerfile
platforms: linux/386,linux/amd64,linux/arm/v7,linux/arm64/v8
push: ${{ github.event_name != 'pull_request' }}
push: true
tags: ${{ steps.docker_meta.outputs.tags }}
labels: ${{ steps.docker_meta.outputs.labels }}

View File

@ -1,5 +1,46 @@
# Changelog
## v3.3.24 (2025/06/04)
* Remove hidden character
## v3.3.23 (2025/06/04)
* Update base image to bookworm. Thanks @rwjack
## v3.3.22 (2025/06/03)
* Disable search engine choice screen
* Fix headless=false stalling. Thanks @MAKMED1337
* Change from click to keys. Thanks @sh4dowb
* Don't open devtools
* Bump Chromium to v137 for build
* Bump requirements
## v3.3.21 (2024/06/26)
* Add challenge selector to catch reloading page on non-English systems
* Escape values for generated form used in request.post. Thanks @mynameisbogdan
## v3.3.20 (2024/06/21)
* maxTimeout should always be int
* Check not running in Docker before logging version_main error
* Update Cloudflare challenge and checkbox selectors. Thanks @tenettow & @21hsmw
## v3.3.19 (2024/05/23)
* Fix occasional headless issue on Linux when set to "false". Thanks @21hsmw
## v3.3.18 (2024/05/20)
* Fix LANG ENV for Linux
* Fix Chrome v124+ not closing on Windows. Thanks @RileyXX
## v3.3.17 (2024/04/09)
* Fix file descriptor leak in service on quit(). Thanks @zkulis
## v3.3.16 (2024/02/28)
* Fix of the subprocess.STARTUPINFO() call. Thanks @ceconelo
* Add FreeBSD support. Thanks @Asthowen
* Use headless configuration properly. Thanks @hashworks
## v3.3.15 (2024/02/20)
* Fix looping challenges

View File

@ -1,4 +1,4 @@
FROM python:3.11-slim-bullseye as builder
FROM python:3.11-slim-bookworm as builder
# Build dummy packages to skip installing them and their dependencies
RUN apt-get update \
@ -12,7 +12,7 @@ RUN apt-get update \
&& equivs-build adwaita-icon-theme \
&& mv adwaita-icon-theme_*.deb /adwaita-icon-theme.deb
FROM python:3.11-slim-bullseye
FROM python:3.11-slim-bookworm
# Copy dummy packages
COPY --from=builder /*.deb /
@ -62,17 +62,17 @@ ENTRYPOINT ["/usr/bin/dumb-init", "--"]
CMD ["/usr/local/bin/python", "-u", "/app/flaresolverr.py"]
# Local build
# docker build -t ngosang/flaresolverr:3.3.15 .
# docker run -p 8191:8191 ngosang/flaresolverr:3.3.15
# docker build -t ngosang/flaresolverr:3.3.24 .
# docker run -p 8191:8191 ngosang/flaresolverr:3.3.24
# Multi-arch build
# docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
# docker buildx create --use
# docker buildx build -t ngosang/flaresolverr:3.3.15 --platform linux/386,linux/amd64,linux/arm/v7,linux/arm64/v8 .
# docker buildx build -t ngosang/flaresolverr:3.3.24 --platform linux/386,linux/amd64,linux/arm/v7,linux/arm64/v8 .
# add --push to publish in DockerHub
# Test multi-arch build
# docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
# docker buildx create --use
# docker buildx build -t ngosang/flaresolverr:3.3.15 --platform linux/arm/v7 --load .
# docker run -p 8191:8191 --platform linux/arm/v7 ngosang/flaresolverr:3.3.15
# docker buildx build -t ngosang/flaresolverr:3.3.24 --platform linux/arm/v7 --load .
# docker run -p 8191:8191 --platform linux/arm/v7 ngosang/flaresolverr:3.3.24

View File

@ -45,7 +45,8 @@ Supported architectures are:
| ARM32 | linux/arm/v7 |
| ARM64 | linux/arm64 |
We provide a `docker-compose.yml` configuration file. Clone this repository and execute `docker-compose up -d` to start
We provide a `docker-compose.yml` configuration file. Clone this repository and execute
`docker-compose up -d` _(Compose V1)_ or `docker compose up -d` _(Compose V2)_ to start
the container.
If you prefer the `docker cli` execute the following command.
@ -78,11 +79,19 @@ This is the recommended way for Windows users.
* Install [Python 3.11](https://www.python.org/downloads/).
* Install [Chrome](https://www.google.com/intl/en_us/chrome/) (all OS) or [Chromium](https://www.chromium.org/getting-involved/download-chromium/) (just Linux, it doesn't work in Windows) web browser.
* (Only in Linux / macOS) Install [Xvfb](https://en.wikipedia.org/wiki/Xvfb) package.
* (Only in Linux) Install [Xvfb](https://en.wikipedia.org/wiki/Xvfb) package.
* (Only in macOS) Install [XQuartz](https://www.xquartz.org/) package.
* Clone this repository and open a shell in that path.
* Run `pip install -r requirements.txt` command to install FlareSolverr dependencies.
* Run `python src/flaresolverr.py` command to start FlareSolverr.
### From source code (FreeBSD/TrueNAS CORE)
* Run `pkg install chromium python311 py311-pip xorg-vfbserver` command to install the required dependencies.
* Clone this repository and open a shell in that path.
* Run `python3.11 -m pip install -r requirements.txt` command to install FlareSolverr dependencies.
* Run `python3.11 src/flaresolverr.py` command to start FlareSolverr.
### Systemd service
We provide an example Systemd unit file `flaresolverr.service` as reference. You have to modify the file to suit your needs: paths, user and environment variables.
@ -95,7 +104,7 @@ curl -L -X POST 'http://localhost:8191/v1' \
-H 'Content-Type: application/json' \
--data-raw '{
"cmd": "request.get",
"url":"http://www.google.com/",
"url": "http://www.google.com/",
"maxTimeout": 60000
}'
```
@ -269,8 +278,8 @@ This is the same as `request.get` but it takes one more param:
Environment variables are set differently depending on the operating system. Some examples:
* Docker: Take a look at the Docker section in this document. Environment variables can be set in the `docker-compose.yml` file or in the Docker CLI command.
* Linux: Run `export LOG_LEVEL=debug` and then start FlareSolverr in the same shell.
* Windows: Open `cmd.exe`, run `set LOG_LEVEL=debug` and then start FlareSolverr in the same shell.
* Linux: Run `export LOG_LEVEL=debug` and then run `flaresolverr` in the same shell.
* Windows: Open `cmd.exe`, run `set LOG_LEVEL=debug` and then run `flaresolverr.exe` in the same shell.
## Prometheus exporter
@ -308,7 +317,7 @@ solve a captcha.
If this is the case, FlareSolverr will return the error `Captcha detected but no automatic solver is configured.`
FlareSolverr can be customized to solve the CAPTCHA automatically by setting the environment variable `CAPTCHA_SOLVER`
to the file name of one of the adapters inside the [/captcha](src/captcha) directory.
to the file name of one of the adapters inside the `/captcha` directory.
## Related projects

19
flaresolverr.service Normal file
View File

@ -0,0 +1,19 @@
[Unit]
Description=FlareSolverr
After=network.target
[Service]
SyslogIdentifier=flaresolverr
Restart=always
RestartSec=5
Type=simple
User=flaresolverr
Group=flaresolverr
Environment="LOG_LEVEL=info"
Environment="CAPTCHA_SOLVER=none"
WorkingDirectory=/opt/flaresolverr
ExecStart=/opt/flaresolverr/flaresolverr
TimeoutStopSec=30
[Install]
WantedBy=multi-user.target

View File

@ -1,6 +1,6 @@
{
"name": "flaresolverr",
"version": "3.3.15",
"version": "3.3.24",
"description": "Proxy server to bypass Cloudflare protection",
"author": "Diego Heras (ngosang / ngosang@hotmail.es)",
"license": "MIT"

View File

@ -1,13 +1,13 @@
bottle==0.12.25
waitress==2.1.2
waitress==3.0.1
selenium==4.15.2
func-timeout==4.3.5
prometheus-client==0.17.1
# required by undetected_chromedriver
requests==2.31.0
certifi==2023.7.22
requests==2.32.3
certifi==2024.07.04
websockets==11.0.3
# only required for linux
xvfbwrapper==0.2.9
# only required for linux and macos
xvfbwrapper==0.2.9; platform_system != "Windows"
# only required for windows
pefile==2023.2.7
pefile==2023.2.7; platform_system == "Windows"

View File

@ -25,7 +25,7 @@ def clean_files():
def download_chromium():
# https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Linux_x64/
revision = "1260008" if os.name == 'nt' else '1260015'
revision = "1453032" if os.name == 'nt' else '1453031'
arch = 'Win_x64' if os.name == 'nt' else 'Linux_x64'
dl_file = 'chrome-win' if os.name == 'nt' else 'chrome-linux'
dl_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), os.pardir, 'dist_chrome')

View File

@ -101,6 +101,9 @@ if __name__ == "__main__":
logging.info(f'FlareSolverr {utils.get_flaresolverr_version()}')
logging.debug('Debug log enabled')
# Get current OS for global variable
utils.get_current_platform()
# test browser installation
flaresolverr_service.test_browser_installation()

View File

@ -3,12 +3,14 @@ import platform
import sys
import time
from datetime import timedelta
from urllib.parse import unquote
from html import escape
from urllib.parse import unquote, quote
from func_timeout import FunctionTimedOut, func_timeout
from selenium.common import TimeoutException
from selenium.webdriver.chrome.webdriver import WebDriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.expected_conditions import (
presence_of_element_located, staleness_of, title_is)
from selenium.webdriver.common.action_chains import ActionChains
@ -40,7 +42,7 @@ CHALLENGE_TITLES = [
]
CHALLENGE_SELECTORS = [
# Cloudflare
'#cf-challenge-running', '.ray_id', '.attack-box', '#cf-please-wait', '#challenge-spinner', '#trk_jschal_js',
'#cf-challenge-running', '.ray_id', '.attack-box', '#cf-please-wait', '#challenge-spinner', '#trk_jschal_js', '#turnstile-wrapper', '.lds-ring',
# Custom CloudFlare for EbookParadijs, Film-Paleis, MuziekFabriek and Puur-Hollands
'td.info #js_info',
# Fairlane / pararius.com
@ -119,7 +121,7 @@ def _controller_v1_handler(req: V1RequestBase) -> V1ResponseBase:
logging.warning("Request parameter 'userAgent' was removed in FlareSolverr v2.")
# set default values
if req.maxTimeout is None or req.maxTimeout < 1:
if req.maxTimeout is None or int(req.maxTimeout) < 1:
req.maxTimeout = 60000
# execute the command
@ -220,7 +222,7 @@ def _cmd_sessions_destroy(req: V1RequestBase) -> V1ResponseBase:
def _resolve_challenge(req: V1RequestBase, method: str) -> ChallengeResolutionT:
timeout = req.maxTimeout / 1000
timeout = int(req.maxTimeout) / 1000
driver = None
try:
if req.session:
@ -245,6 +247,8 @@ def _resolve_challenge(req: V1RequestBase, method: str) -> ChallengeResolutionT:
raise Exception('Error solving the challenge. ' + str(e).replace('\n', '\\n'))
finally:
if not req.session and driver is not None:
if utils.PLATFORM_VERSION == "nt":
driver.close()
driver.quit()
logging.debug('A used instance of webdriver has been destroyed')
@ -252,18 +256,9 @@ def _resolve_challenge(req: V1RequestBase, method: str) -> ChallengeResolutionT:
def click_verify(driver: WebDriver):
try:
logging.debug("Try to find the Cloudflare verify checkbox...")
iframe = driver.find_element(By.XPATH, "//iframe[starts-with(@id, 'cf-chl-widget-')]")
driver.switch_to.frame(iframe)
checkbox = driver.find_element(
by=By.XPATH,
value='//*[@id="challenge-stage"]/div/label/input',
)
if checkbox:
actions = ActionChains(driver)
actions.move_to_element_with_offset(checkbox, 5, 7)
actions.click(checkbox)
actions.perform()
logging.debug("Cloudflare verify checkbox found and clicked!")
actions = ActionChains(driver)
actions.pause(5).send_keys(Keys.TAB).pause(1).send_keys(Keys.SPACE).perform()
logging.debug("Cloudflare verify checkbox found and clicked!")
except Exception:
logging.debug("Cloudflare verify checkbox not found on the page.")
finally:
@ -287,22 +282,6 @@ def click_verify(driver: WebDriver):
time.sleep(2)
def get_correct_window(driver: WebDriver) -> WebDriver:
if len(driver.window_handles) > 1:
for window_handle in driver.window_handles:
driver.switch_to.window(window_handle)
current_url = driver.current_url
if not current_url.startswith("devtools://devtools"):
return driver
return driver
def access_page(driver: WebDriver, url: str) -> None:
driver.get(url)
driver.start_session()
driver.start_session() # required to bypass Cloudflare
def _evil_logic(req: V1RequestBase, driver: WebDriver, method: str) -> ChallengeResolutionT:
res = ChallengeResolutionT({})
res.status = STATUS_OK
@ -314,8 +293,7 @@ def _evil_logic(req: V1RequestBase, driver: WebDriver, method: str) -> Challenge
if method == 'POST':
_post_request(req, driver)
else:
access_page(driver, req.url)
driver = get_correct_window(driver)
driver.get(req.url)
# set cookies if required
if req.cookies is not None and len(req.cookies) > 0:
@ -327,8 +305,7 @@ def _evil_logic(req: V1RequestBase, driver: WebDriver, method: str) -> Challenge
if method == 'POST':
_post_request(req, driver)
else:
access_page(driver, req.url)
driver = get_correct_window(driver)
driver.get(req.url)
# wait for the page
if utils.get_config_log_html():
@ -437,7 +414,7 @@ def _post_request(req: V1RequestBase, driver: WebDriver):
value = unquote(parts[1])
except Exception:
value = parts[1]
post_form += f'<input type="text" name="{name}" value="{value}"><br>'
post_form += f'<input type="text" name="{escape(quote(name))}" value="{escape(quote(value))}"><br>'
post_form += '</form>'
html_content = f"""
<!DOCTYPE html>
@ -447,6 +424,4 @@ def _post_request(req: V1RequestBase, driver: WebDriver):
<script>document.getElementById('hackForm').submit();</script>
</body>
</html>"""
driver.get("data:text/html;charset=utf-8," + html_content)
driver.start_session()
driver.start_session() # required to bypass Cloudflare
driver.get("data:text/html;charset=utf-8,{html_content}".format(html_content=html_content))

View File

@ -66,6 +66,8 @@ class SessionsStorage:
return False
session = self.sessions.pop(session_id)
if utils.PLATFORM_VERSION == "nt":
session.driver.close()
session.driver.quit()
return True

View File

@ -451,8 +451,10 @@ class Chrome(selenium.webdriver.chrome.webdriver.WebDriver):
options.binary_location, *options.arguments
)
else:
startupinfo = subprocess.STARTUPINFO()
startupinfo = None
if os.name == 'nt' and windows_headless:
# STARTUPINFO() is Windows only
startupinfo = subprocess.STARTUPINFO()
startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
browser = subprocess.Popen(
[options.binary_location, *options.arguments],
@ -505,8 +507,6 @@ class Chrome(selenium.webdriver.chrome.webdriver.WebDriver):
"Page.addScriptToEvaluateOnNewDocument",
{
"source": """
Object.defineProperty(window, "navigator", {
Object.defineProperty(window, "navigator", {
value: new Proxy(navigator, {
has: (target, key) => (key === "webdriver" ? false : key in target),
@ -769,7 +769,9 @@ class Chrome(selenium.webdriver.chrome.webdriver.WebDriver):
def quit(self):
try:
self.service.stop()
self.service.process.kill()
self.command_executor.close()
self.service.process.wait(5)
logger.debug("webdriver process ended")
except (AttributeError, RuntimeError, OSError):
@ -784,15 +786,6 @@ class Chrome(selenium.webdriver.chrome.webdriver.WebDriver):
logger.debug("gracefully closed browser")
except Exception as e: # noqa
pass
# Force kill Chrome process in Windows
# https://github.com/FlareSolverr/FlareSolverr/issues/772
if os.name == 'nt':
try:
subprocess.call(['taskkill', '/f', '/pid', str(self.browser_pid)],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL)
except Exception:
pass
if (
hasattr(self, "keep_user_data_dir")
and hasattr(self, "user_data_dir")

View File

@ -2,6 +2,7 @@ import asyncio
from collections.abc import Mapping
from collections.abc import Sequence
from functools import wraps
import os
import logging
import threading
import time
@ -187,4 +188,6 @@ def test():
time.sleep(10)
if os.name == "nt":
driver.close()
driver.quit()

View File

@ -21,7 +21,7 @@ from multiprocessing import Lock
logger = logging.getLogger(__name__)
IS_POSIX = sys.platform.startswith(("darwin", "cygwin", "linux", "linux2"))
IS_POSIX = sys.platform.startswith(("darwin", "cygwin", "linux", "linux2", "freebsd"))
class Patcher(object):
@ -68,8 +68,10 @@ class Patcher(object):
# check if version_main_int is less than or equal to e.g 114
self.is_old_chromedriver = version_main and version_main_int <= 114
except (ValueError,TypeError):
# If the conversion fails, print an error message
print("version_main cannot be converted to an integer")
# Check not running inside Docker
if not os.path.exists("/app/chromedriver"):
# If the conversion fails, log an error message
logging.info("version_main cannot be converted to an integer")
# Set self.is_old_chromedriver to False if the conversion fails
self.is_old_chromedriver = False
@ -80,9 +82,14 @@ class Patcher(object):
os.makedirs(self.data_path, exist_ok=True)
if not executable_path:
self.executable_path = os.path.join(
self.data_path, "_".join([prefix, self.exe_name])
)
if sys.platform.startswith("freebsd"):
self.executable_path = os.path.join(
self.data_path, self.exe_name
)
else:
self.executable_path = os.path.join(
self.data_path, "_".join([prefix, self.exe_name])
)
if not IS_POSIX:
if executable_path:
@ -127,6 +134,9 @@ class Patcher(object):
else:
self.platform_name = "mac-x64"
self.exe_name %= ""
if self.platform.startswith("freebsd"):
self.platform_name = "freebsd"
self.exe_name %= ""
def auto(self, executable_path=None, force=False, version_main=None, _=None):
"""
@ -166,26 +176,56 @@ class Patcher(object):
if force is True:
self.force = force
try:
os.unlink(self.executable_path)
except PermissionError:
if self.force:
self.force_kill_instances(self.executable_path)
return self.auto(force=not self.force)
try:
if self.is_binary_patched():
# assumes already running AND patched
return True
except PermissionError:
pass
# return False
except FileNotFoundError:
pass
release = self.fetch_release_number()
self.version_main = release.version[0]
self.version_full = release
self.unzip_package(self.fetch_package())
if self.platform_name == "freebsd":
chromedriver_path = shutil.which("chromedriver")
if not os.path.isfile(chromedriver_path) or not os.access(chromedriver_path, os.X_OK):
logging.error("Chromedriver not installed!")
return
version_path = os.path.join(os.path.dirname(self.executable_path), "version.txt")
process = os.popen(f'"{chromedriver_path}" --version')
chromedriver_version = process.read().split(' ')[1].split(' ')[0]
process.close()
current_version = None
if os.path.isfile(version_path) or os.access(version_path, os.X_OK):
with open(version_path, 'r') as f:
current_version = f.read()
if current_version != chromedriver_version:
logging.info("Copying chromedriver executable...")
shutil.copy(chromedriver_path, self.executable_path)
os.chmod(self.executable_path, 0o755)
with open(version_path, 'w') as f:
f.write(chromedriver_version)
logging.info("Chromedriver executable copied!")
else:
try:
os.unlink(self.executable_path)
except PermissionError:
if self.force:
self.force_kill_instances(self.executable_path)
return self.auto(force=not self.force)
try:
if self.is_binary_patched():
# assumes already running AND patched
return True
except PermissionError:
pass
# return False
except FileNotFoundError:
pass
release = self.fetch_release_number()
self.version_main = release.version[0]
self.version_full = release
self.unzip_package(self.fetch_package())
return self.patch()
def driver_binary_in_use(self, path: str = None) -> bool:

View File

@ -1,15 +1,18 @@
import json
import logging
import os
import platform
import re
import shutil
import urllib.parse
import sys
import tempfile
import urllib.parse
from selenium.webdriver.chrome.webdriver import WebDriver
import undetected_chromedriver as uc
FLARESOLVERR_VERSION = None
PLATFORM_VERSION = None
CHROME_EXE_PATH = None
CHROME_MAJOR_VERSION = None
USER_AGENT = None
@ -37,6 +40,13 @@ def get_flaresolverr_version() -> str:
FLARESOLVERR_VERSION = json.loads(f.read())['version']
return FLARESOLVERR_VERSION
def get_current_platform() -> str:
global PLATFORM_VERSION
if PLATFORM_VERSION is not None:
return PLATFORM_VERSION
PLATFORM_VERSION = os.name
return PLATFORM_VERSION
def create_proxy_extension(proxy: dict) -> str:
parsed_url = urllib.parse.urlparse(proxy['url'])
@ -120,14 +130,17 @@ def get_webdriver(proxy: dict = None) -> WebDriver:
options = uc.ChromeOptions()
options.add_argument('--no-sandbox')
options.add_argument('--window-size=1920,1080')
options.add_argument('--disable-search-engine-choice-screen')
# todo: this param shows a warning in chrome head-full
options.add_argument('--disable-setuid-sandbox')
options.add_argument('--disable-dev-shm-usage')
# this option removes the zygote sandbox (it seems that the resolution is a bit faster)
options.add_argument('--no-zygote')
# attempt to fix Docker ARM32 build
options.add_argument('--disable-gpu-sandbox')
options.add_argument('--disable-software-rasterizer')
IS_ARMARCH = platform.machine().startswith(('arm', 'aarch'))
if IS_ARMARCH:
options.add_argument('--disable-gpu-sandbox')
options.add_argument('--disable-software-rasterizer')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
# fix GL errors in ASUSTOR NAS
@ -138,7 +151,7 @@ def get_webdriver(proxy: dict = None) -> WebDriver:
language = os.environ.get('LANG', None)
if language is not None:
options.add_argument('--lang=%s' % language)
options.add_argument('--accept-lang=%s' % language)
# Fix for Chrome 117 | https://github.com/FlareSolverr/FlareSolverr/issues/910
if USER_AGENT is not None:
@ -164,8 +177,6 @@ def get_webdriver(proxy: dict = None) -> WebDriver:
# For normal headless mode:
# options.add_argument('--headless')
options.add_argument("--auto-open-devtools-for-tabs")
# if we are inside the Docker container, we avoid downloading the driver
driver_exe_path = None
version_main = None
@ -182,9 +193,12 @@ def get_webdriver(proxy: dict = None) -> WebDriver:
# downloads and patches the chromedriver
# if we don't set driver_executable_path it downloads, patches, and deletes the driver each time
driver = uc.Chrome(options=options, browser_executable_path=browser_executable_path,
driver_executable_path=driver_exe_path, version_main=version_main,
windows_headless=windows_headless, headless=windows_headless)
try:
driver = uc.Chrome(options=options, browser_executable_path=browser_executable_path,
driver_executable_path=driver_exe_path, version_main=version_main,
windows_headless=windows_headless, headless=get_config_headless())
except Exception as e:
logging.error("Error starting Chrome: %s" % e)
# save the patched driver to avoid re-downloads
if driver_exe_path is None:
@ -310,6 +324,8 @@ def get_user_agent(driver=None) -> str:
raise Exception("Error getting browser User-Agent. " + str(e))
finally:
if driver is not None:
if PLATFORM_VERSION == "nt":
driver.close()
driver.quit()