github/FlareSolverr
1
1
Fork 0
mirror of https://github.com/FlareSolverr/FlareSolverr.git synced 2025-06-08 12:35:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
340 Commits 5 Branches 64 Tags 2 MiB
Commit Graph

173 Commits

Author SHA1 Message Date
ngosang
66b9db21e5 Fix Chromium exec permissions in Linux package 2023-05-21 21:09:06 +02:00
ngosang
ac77110578 Remove redundant artifact from Windows binary package 2023-04-02 21:04:34 +02:00
Rawand Ahmed Shaswar
a9d1a2de2d
Fix headless mode in mac OS (#750) 2023-04-02 20:49:18 +02:00
ngosang
e0bf02fb8b Distribute binary executables in compressed package 2023-03-25 22:19:26 +01:00
ilike2burnthing
82a1cd835a
Add icon for binary executable (#739) 2023-03-25 21:35:37 +01:00
ngosang
ae18559db1 Check Python version on start 2023-03-25 21:23:06 +01:00
ngosang
8d9bac9dd4 Build binaries for Linux x64 and Windows x64 2023-03-20 22:30:52 +01:00
ngosang
30ccf18e85 Several fixes in Sessions 2023-03-20 17:06:16 +01:00
ngosang
a15d041a0c Fix Waitress server error with asyncore_use_poll=true. Resolves #680 2023-03-20 16:27:25 +01:00
Martino Mensio
c6c74e7c9d Add Fairlane challenge selector (#723) 2023-03-20 16:27:24 +01:00
Artemiy Ryabinkov
49fd1aacfc
Sessions with auto-creation on fetch request and TTL (#736) 
* Add support for sessions

* Add tests for sessions

* Missing return type

* Don't re-create an existing session

* Return success in case of session doesn't exists on destroy

* Create session if necessary on get request

* Add session TTL to the request.get method

When fetching some webpage with a predefined session id,
FlareSorverr is using existing instance of WebDriver.
That allows user to not manage cookies explicitly
and rely on WebDriver to maintain the session. However,
if session has been created long time ago, CloudFlare might
stop accepting the requests, so we want to recreate the session
time to time. From the user perspective the easiest way of doing it
is to define their expectation on the session duration.

These changes add an option to define Time-to-live (TTL) for the session
and FlareSorverr takes care about rotating the sessions.

* Update message for session destroy in tests

---------

Co-authored-by: Michel Roux <xefir@crystalyx.net>
 2023-03-20 16:25:48 +01:00
Martino Mensio
f6879c70de
Add Fairlane challenge selector (#723) 2023-03-20 16:13:21 +01:00
ngosang
24f59a39cb Print platform information on start up 2023-03-20 15:07:12 +01:00
ngosang
8de16058d0 Attempt to fix Docker ARM32 build 2023-03-20 15:07:12 +01:00
bilditup1
5fc4f966a5
Update ddos-guard title (#692) 
* update ddos-guard title (anidex.info)

* make page_title test case-insensitive

---------

Co-authored-by: bilditup1 <git@github.com>
 2023-03-20 15:06:27 +01:00
ilike2burnthing
fc6d2d9095
Click on the Cloudflare's 'Verify you are human' button if necessary 
Co-authored-by: furdarius <furdarius@users.noreply.github.com>

69e023b946 minus screenshot function
 2023-03-07 03:51:17 +00:00
Artemiy Ryabinkov
96fcd21174
Update undetected_chromedriver version to 3.4.6 (#715) 
Co-authored-by: ilike2burnthing <59480337+ilike2burnthing@users.noreply.github.com>
 2023-03-06 13:57:38 +00:00
ngosang
c93834e2f0 Check Chrome / Chromium web browser is installed correctly 2023-01-08 20:46:11 +01:00
ngosang
e3b4200d94 Detect Cloudflare blocked access 2023-01-08 20:40:10 +01:00
ngosang
2aa1744476 Add more selectors to detect blocked access 2023-01-06 18:12:26 +01:00
ngosang
a89679a52d Disable Zygote sandbox in Chromium browser 2023-01-06 18:05:23 +01:00
ngosang
410ee7981f Apply undetected-chromedriver patches 
* Hide Chrome window in Windows/NT
* Not use subprocess by default (independent process)
* Kill Chromium processes properly to avoid defunct/zombie processes
 2023-01-06 17:50:52 +01:00
ngosang
e163019f28 Update undetected-chromedriver 2023-01-06 17:19:11 +01:00
ngosang
7d84f1b663 Kill Chromium processes properly to avoid defunct/zombie processes 2023-01-06 13:58:24 +01:00
ngosang
fd42fcee66 Detect challenges by title 2023-01-05 01:45:43 +01:00
ngosang
258225a091 Detect Cloudflare access denied 2023-01-04 22:15:19 +01:00
ngosang
70a423805d Fix selector for new challenge 2023-01-04 21:28:04 +01:00
ngosang
676a930f02 Reuse patched chromedriver 2023-01-04 21:05:32 +01:00
ngosang
ecaac2e1d9 Fix Chrome version detection on Windows 2023-01-04 21:05:27 +01:00
ngosang
60a22625be Add browser headless mode for Windows 2023-01-04 21:05:19 +01:00
ngosang
b32d7b70be Fork undetected-chromedriver 3.1.5.post4 2023-01-04 21:05:12 +01:00
ngosang
5092b1eef9 Add browser headless mode for Linux 2023-01-04 21:04:38 +01:00
ngosang
0cfcb57e3a Add tests for several known sites 2023-01-04 21:04:31 +01:00
ngosang
59bd7c8bdd Show ReqId only in Debug traces 2023-01-04 21:04:26 +01:00
ngosang
b90d263159 Detect Cloudflare Access Denied 2023-01-04 21:04:18 +01:00
ngosang
1505595591 Rewrite FlareSolverr from scratch in Python + Selenium 2023-01-04 21:03:40 +01:00
ngosang
383025032b Prepare for version 3.0, remove JS code 2023-01-04 20:57:23 +01:00
Athorcis
197258e921
detect DDos-Guard through title content. resolves #546 (#559) 2022-10-22 14:34:29 +01:00
ngosang
aa254eb830 Detect Cloudflare Access Denied 2022-09-25 00:35:39 +02:00
ngosang
b17a3a369b Remove 30 s delay and clean legacy code 2022-09-17 23:48:39 +02:00
ilike2burnthing
0e2452e40e
temp fix: add 30s delay 
credit: @realivanjx
https://github.com/FlareSolverr/FlareSolverr/issues/481#issuecomment-1242951929
 2022-09-12 01:44:01 +01:00
ngosang
a2345affb3 Fix Cloudflare detection in POST requests 2022-07-31 16:06:28 +02:00
ngosang
c5df58529a Update Cloudflare selectors and add HTML samples 2022-07-30 22:36:51 +02:00
ngosang
d962e1a14e Detect DDoS-Guard challenge 2022-04-17 09:21:10 +02:00
ngosang
d34b43e0a8 Fix 2000 ms navigation timeout 2022-04-16 21:39:50 +02:00
ngosang
9b1f8332c7 Disable Cloudflare ban detection 2022-04-16 18:32:58 +02:00
Harold
bb4fa9cabc
Fix ban detection. Resolves #330 (#336) 2022-03-19 04:24:49 +00:00
ngosang
6c598d5360 Fix max timeout error in some pages 2022-02-06 16:35:52 +01:00
ngosang
2893f72237 Avoid crashing in NodeJS 17 due to Unhandled promise rejection 2022-02-06 13:31:30 +01:00
ngosang
cd221bbbf1 Improve proxy validation and debug traces 2022-02-06 13:07:11 +01:00
ngosang
56fc688517 Increase default BROWSER_TIMEOUT=40000 (40 seconds) 2022-01-30 23:24:15 +01:00
ngosang
0a438358d1 Fix Puppeter deprecation warnings 2022-01-30 23:23:06 +01:00
ngosang
fe6cfd75b8 Update Puppeter and other dependencies 2022-01-30 22:49:15 +01:00
ngosang
bb7e82e6c4 Add support for Custom CloudFlare challenge 
EbookParadijs, Film-Paleis, MuziekFabriek and Puur-Hollands
 2022-01-30 21:32:16 +01:00
ngosang
fdd1d245f4 Add support for DDoS-GUARD challenge 2022-01-30 20:36:38 +01:00
simonfr
a9ab2569bc
Add aarch64 to user agents to be replaced (#248) 
Co-authored-by: Simon <simon@perols.dev>
 2021-12-12 16:46:20 +01:00
ngosang
b1a6ad7688 Fix SOCKSv4 and SOCKSv5 proxy. resolves #214 #220 2021-12-12 14:29:38 +01:00
ngosang
c4ef6a472e Make test URL configurable with TEST_URL env var. resolves #240 2021-12-12 12:35:05 +01:00
ngosang
a24b665bd1 Bypass new Cloudflare protection 2021-12-12 12:35:05 +01:00
ngosang
3005ba3629 Fix SOCKS5 proxy. Resolves #214 2021-10-31 22:39:32 +01:00
ngosang
7a1cf7dd80 Catch startup exceptions and give some advices 2021-10-31 22:12:55 +01:00
ngosang
456dfc222e Add env var BROWSER_TIMEOUT for slow systems 2021-10-31 21:56:25 +01:00
ngosang
47c83ded58 Check user home dir before testing web browser installation 2021-10-24 15:52:03 +02:00
ngosang
753e8e1be8 Set puppeteer timeout half of maxTimeout param. Resolves #180 2021-10-20 18:28:30 +02:00
ngosang
a6628d0cda Add test for blocked IP 2021-10-20 18:06:25 +02:00
ngosang
a79a5f2b42 Avoid reloading the page in case of error 2021-10-20 18:06:15 +02:00
ngosang
1e463bb3e2 Improve Cloudflare detection 2021-10-20 18:05:59 +02:00
ngosang
02204a84d3 Fix version 2021-10-20 03:39:24 +02:00
ngosang
95d178b37a Fix browser preferences and proxy 2021-10-20 01:00:54 +02:00
ngosang
c4f890f9a1 Fix request.post method and clean error traces 2021-10-20 01:00:35 +02:00
ngosang
d16b982bb9 Use Firefox ESR for Docker images 2021-10-18 22:52:38 +02:00
ngosang
075b53ee24 Improve Firefox start time and code clean up 2021-10-18 21:45:21 +02:00
ngosang
356b893c18 Improve bad request management and tests 2021-10-18 19:27:21 +02:00
ngosang
a841d67745 Build native packages with Firefox 2021-10-18 11:13:49 +02:00
ngosang
77a87c79fd Improve Docker image and clean TODOs 2021-10-18 00:23:28 +02:00
ngosang
cfd158462f Add proxy support 2021-10-18 00:02:30 +02:00
ngosang
ccfe21c15a Implement request.post method for Firefox 2021-10-17 22:05:15 +02:00
ngosang
a5b3e08e1f Code clean up, remove returnRawHtml, download, headers params 2021-10-17 20:43:36 +02:00
ngosang
a0e897067a Remove outdated chaptcha solvers 2021-10-17 18:25:44 +02:00
ngosang
744de4d158 Refactor the app to use Express server and Jest for tests 2021-10-17 18:00:19 +02:00
ngosang
0459f2642d Fix Cloudflare resolver for Linux ARM builds 2021-10-16 20:26:10 +02:00
ngosang
ca3f84f458 Fix Cloudflare resolver 2021-10-16 19:32:52 +02:00
ngosang
5dd563e003 Replace Chrome web browser with Firefox 2021-10-16 19:16:25 +02:00
ngosang
78c10d6b24 Remove userAgent parameter since any modification is detected by CF 2021-10-16 17:46:04 +02:00
ngosang
7738f7a360 Remove Puppeter steath plugin 2021-10-16 16:54:56 +02:00
ngosang
447c8f67a1 Improve "Execution context was destroyed" error handling 2021-08-01 22:10:53 +02:00
ngosang
9dae74bc28 Implement returnRawHtml parameter. resolves #172 resolves #165 2021-08-01 22:08:55 +02:00
ngosang
4199db5a41 Capture Docker stop signal. resolves #158 2021-08-01 21:37:45 +02:00
ngosang
232ddca512 Fix page reload after challenge is solved. resolves #162 resolves #143 2021-08-01 20:34:38 +02:00
ngosang
8572fab781 Avoid loading images/css/fonts to speed up page load 2021-08-01 19:35:26 +02:00
ngosang
fdb3eae051 Improve Cloudflare IP ban detection 2021-08-01 19:32:09 +02:00
ngosang
543ce89eb6 Improve old JS challenge waiting. Resolves #129 2021-06-01 01:59:57 +02:00
ngosang
24f1b4ec6f Improvements in Cloudflare redirect detection. Resolves #140 2021-06-01 01:21:06 +02:00
ngosang
5242cf3359 Show an error in hcaptcha-solver. Resolves #132 2021-05-30 14:15:08 +02:00
ngosang
c6677f4d84 Handle new Cloudflare challenge. Resolves #135 Resolves #134 2021-05-30 13:40:17 +02:00
ngosang
5da5156851 Fix EACCES: permission denied, open '/tmp/flaresolverr.txt'. Resolves #120 2021-05-30 11:38:20 +02:00
ngosang
05f8ef95d9 Configure timezone with TZ env var. Resolves #109 2021-05-30 11:28:43 +02:00
Arias800
6cf948d0e1
Return the redirected URL in the response (#126) 
It adds the possibility for the user to get the final url after a redirection.
 2021-05-30 09:29:21 +02:00
ngosang
05abe69df6 Fix memory regression, close test browser 2021-04-05 05:26:45 +02:00
ngosang
a85e9c2c8c Validate Chrome is working at startup 2021-04-04 22:37:53 +02:00