AdGuard 5.10 second beta version release

[avatar]

Hello everyone!

We are finally ready to release second beta version of Adguard 5.10.

Changelog:

What’s new?

+ Added capability to add known ABP filter subscriptions.
In addition to our own filters, you can now add known ABP subscriptions, such as EasyList and its supplementary subscription and others.

+ Massively redesigned work with extensions.
Changed approach to storage of extensions. This will give us the opportunity to add a userscripts edit function in the next beta. Note: Due to this change, the list of extensions will be reset to the state by default after the update.

+ Integration with Web of Trust is separated out to the stand-alone extension that can be disabled.

+ Added support for installing extensions from all known public repositories.
This means that there is no need in giving a link to a userscript itself, a link to its page on the corresponding repository will be enough.
Page with description of repositories: http://adguard.com/userscripts.html

+ Added support for Superbird Browser.

+ Added support for 360 Browser.

Fixed and changed:

* Conflict with Keepass.
* Conflict with Visual Studio (Adguard hindered the debugging).
* Fixed work of Twitch Chat Filter userscript.
* Partially fixed conflict with the Tor Browser. HTTP-sites are filtered, HTTPS – are not.
* Fixed update mechanism of userscripts.
* Updates are checked only for enabled userscripts.
* Fixed a bug with the proxy authorization.
* Fixed compatibility of vk.com and our Parental control.
* Fixed defect on disconnecting Adguard Assistant via the context menu.
* Increased the filter update period up to 12 hours.
* Updated network drivers, some small issues resolved.
* Fixed some minor defects of Adguard Assistant.
* Minor fixes to the texts in the user interface.
* Accelerated the work of extensions (including Youtube Center).


What will be added to the next beta versions:

+ Section "White List".
Exceptions for HTTPS-filtering and Browsing Security can be configured there.

+ Ability to edit and export the userscripts.

UPD:
Next beta-version is here.

Changelog for version 5.10.1128.5771:

* Fixed "databases have not been loaded" issue
* Fixed user filter rules edit
* Fixed userscripts API thanks to YePpHa from YTC
* Fixed userscripts update check
* Fixed tray menu
* Fixed bug with "Show useful ads" checkbox
* Fixed filters update check
* Fixed misspelling in UI

UPD2:
Changelog for version 5.10.1136.5808:

* The situation when UI hangs in "Preparing to launch" screen
* Random crashes on user filters edit
* Added SSL exceptions setting
* Added userscripts validation on install
* Added "Reload" option to every userscript
* Fixed HTTP DELETE requests
* Google SafeBrowisng removed from application

UPD3:
Version 5.10.1144.5852 is available now.

What's new:

+ Now you can edit userscripts source code

Fixed:

* Fixed work of URL-rules with "~third-party" modifier
* Fixed UI crashes on user filter edit (this time it is really fixed)
* Speed up filtering of the web pages which use large javascript files (originally it is octane-benchmark.googlecode.com issue)
* Fixed Tor Browser issue
* Fixed "Filters databases have not been loaded" message on UI startup
* Fixed several minor bugs

UPD4:
New beta-version is here.
It is almost release candidate. The single major bug is @Rian's issue with constant page loading.

What's new in 5.10.1148.5881:

+ Updated Web of Trust extension.
New warning popup UI. Also Adguard now detects if WOT browser extension is installed and does not show it's warning in that case.
+ New TDI and WPF driver. Fixed some minor compatiblity issues.

Fixed:
* Extension's "Reload" does not block UI anymore
* Fixed issue with skipping Application configuration on the first run
* Fixed compatibility issue with Adguard Assistant and requirejs

UPD5:
Release candidate is ready.

What's new in version 5.10.1152.5901:

* Fixed some minor defects of Web of Trust extension
* Fixed Assistant's bug with selecting complex elements (like those on howtogeeks.com)
* While updating from Adguard 5.9 Adguard migrates some of userscripts (for instance, Youtube Center should be migrated to 5.10 successfully)

UPD6:
Second release candidate is ready. Version 5.10.1154.5920.

Changelog:
* Fixed "Blocked by Adguard" issues
* Fixed bug with userscripts update from 5.9 to 5.10
* Minor bugfixes

If no important bugs are found, this will be the final 5.10 release.

UPD7:
Next RC is ready (i hope it is the last RC). Version 5.10.1156.5925.

Changelog:
* Fixed constant page load in PaleMoon
* Fixed "Go back" button in WOT extension

UPD8:
A few more fixes brings version 5.10.1173.6059.

Changelog for version 5.10.1173.6059:

+ New version of our network driver (minor bugfixes)
* Fixed bug with Google Chrome switching to QUIC protocol
* Fixed bug of Browsing Security with checking fifth-level domains
* Fixed some compatibility issues with some userscripts (added some more GM_ functions: GM_setClipboard, GM_listValues, GM_deleteValue)
* Fixed some app crashes
* Fixed bug with some userscripts updates
* Fixed bug of Assistant with adding new rules on the websites which use aggressive caching
* Fixed bug in Parental Control: ignoring blacklist if Sensitivity level is set to disabled

UPD 9
Meanwhile new version - new fixes:

Changelog for Version 5.10.1175.6083

* Teredo is no more disabled while using TDI.
* Fixed error messages in IE8.
* Added new GreaseMonkey emulation functions.
* Fixed Adguard pop-up messages localization

UPD 10:
New version 5.10.1178.6114 brings a couple of improvements

Changelog:

* Updated network drivers
* Fixed a problem with the unavailability of websites

UPD 11:
Some more fixes in new 5.10.1180.6137 version

Changelog:

* Parental Control blocks access to unsafe websites (phishing, malware)
* Added support for regexp rules
* Updated network drivers
* Modified filtering mechanism
* Minor localization fixes

UPD 12:
Version 5.10.1183.6160 brings few changes

Changelog:

- Maxthon Nitro added to the list of filtered browsers
- Cosmetic changes

UPD 13:
Release candidate for second Adguard 5.10 patch is here, version 5.10.1185.6167.

Full list of changes from the last release:

Added:

+ New version of Adguard network drivers
+ Added Maxthon Nitro support
+ Added support for Regexp-rules
+ Added new API GreaseMonkey functions for better userscripts support

Fixed:

* Fixed an issue with a long delay when adding rules by Assistant
* Now, even when the Parental Control sensitivity level is disabled, black and white lists continue to work
* Fixed program crashes when working with tray
* When using TDI driver Teredo is no longer disabled
* Fixed bug reports in IE8
* Fixed several localization bugs
* Parental control now blocks access to unsafe sites even when the sensitivity level is disabled

 

[Boo Berry]

Noticed right off the bat, YouTube Center doesn't seem to work anymore - tested in all browsers including IE. I'll do clean profiles and re-test.

Using this link for the latest development builds: https://github.com/YePpHa/YouTubeCenter/raw/master/dist/YouTubeCenter.user.js

 

[avatar]

Noticed right off the bat, YouTube Center doesn't seem to work anymore - tested in all browsers including IE. I'll do clean profiles and re-test.

Using this link for the latest development builds: https://github.com/YePpHa/YouTubeCenter/raw/master/dist/YouTubeCenter.user.js

Use previous developer build. They have broken something in the last build.

---------- Post added at 09:41 PM ---------- Previous post was at 09:38 PM ----------

There is a link:
https://github.com/YePpHa/YouTubeCenter/raw/3d4711eea242d7b6a2a38fe221422b2e3c654c88/dist/YouTubeCenter.user.js

 

[Boo Berry]

Probably this change: https://github.com/YePpHa/YouTubeCenter/commit/343b4fdf169a6c300597f97b016f2d621d22a639

In the meantime I'll use Tampermonkey/Greasemonkey for YTC.

---------- Post added at 01:01 PM ---------- Previous post was at 12:42 PM ----------

The widget.banner-square element on Neowin.net isn't being blocked anymore.

---------- Post added at 02:32 PM ---------- Previous post was at 01:01 PM ----------

Enabling EasyList might conflict with the Adguard filter lists if both enabled at the same time.

 

[mysteriously]

Thanks for the release. Is there any way to add custom (unlisted) Adblock filter list? I would like to add one from:
http://adblocklist.org/

@Petrovic I saw this and its not what I'm taking about. Yes, we can choose additional filters from the list, but we don't have ability to use other filter list (non shown there).
Example URL:
https://dl.dropboxusercontent.com/u/25725152/cookiesblock.txt

Using Adblock Plus as web browser extension you can subscribe any URL filter list you want.

 

[Petrovic]

Thanks for the release. Is there any way to add custom (unlisted) Adblock filter list? I would like to add one from:
http://adblocklist.org/

 

[rickyoh]

looking good. like the expanded filter selection menu i'm spoilt for choice! now on to some observations:

- seeing a "Filter databases have not been loaded" message on the start page

Spoiler

- misspelling of subscription in the filters subscriptions dialog

Spoiler

also is it right to have it as "filters subscriptions" or can "filter subscriptions" also work ?

- clicking the "Wiltteri" homepage shortcut in the filter subscriptions dialog leads to a Google search of "wiltteri.net". the proper website, it appears, is

https://github.com/wiltteri/wiltteri

- anti-adblock killer does not seem to work
- any url can be input in the "Extension installation" dialog resulting in a successful installation with the url as the extension name and 1.0.0.0 as its version(try it with this thread's url) can confirm that an installation is also successful should you browse to any file renamed to xxxxx.user.js(an image file in my case)

also not a bug but a suggestion;
-since the "Adblock Warning Removal List" is part of the Easylist group of filters, wouldn't it make sense to have it appear under EasyList ?
- shouldn't the homepage link next to the "User filter" entry in the Ad Blocker section load the user filter section of Adguard rather than Adguard's homepage ?

 

[Boo Berry]

YouTube Center Developer Version is working again in Adguard (after I reported the issue) if you use 339 or above.

But...

I noticed that YouTube Center's settings (I assume local storage or something else) is being saved globally between all browsers, and not for each specific browser like it was doing not too long ago. This can be a problem as I prefer to force HTML5 in Chrome, while forcing Flash in Chromium (due to the lack of H.264 support in Chromium). Same with Firefox and Nightly, I prefer to force Flash for Firefox (until at least FF31) and HTML5 for Nightly. Is this as intended? Is it a bug or an oversight? Is it an issue with the script itself?

---------- Post added at 06:31 PM ---------- Previous post was at 05:52 PM ----------

This was posted here: https://github.com/YePpHa/YouTubeCenter/issues/639#issuecomment-48548421

Seems like Adguard has a very major security issue. Any website you visit can use the GM API or Adguard's version of the GM API. Example a website with the domain: www.example.com tries to make a request to this page: https://raw.github.com/YePpHa/YouTubeCenter/master/devbuild.number will just have to use Adguard's proxy that is available to every domain like:
http://www.example.com/adguard-ajax-api/api?type=gm-xml-http-request&url=https://raw.github.com/YePpHa/YouTubeCenter/master/devbuild.number&data=&headers=Content-Type:text/plain&method=GET&overridemimetype=&user=&password=

The domain of the proxy can be anything:
http://www.google.com/adguard-ajax-api/api?type=gm-xml-http-request&url=https://raw.github.com/YePpHa/YouTubeCenter/master/devbuild.number&data=&headers=Content-Type:text/plain&method=GET&overridemimetype=&user=&password=
or
http://www.runerne.dk/adguard-ajax-api/api?type=gm-xml-http-request&url=https://raw.github.com/YePpHa/YouTubeCenter/master/devbuild.number&data=&headers=Content-Type:text/plain&method=GET&overridemimetype=&user=&password=

They will all access https://raw.github.com/YePpHa/YouTubeCenter/master/devbuild.number. This could potentially be very insecure in that way that malicious website could steal information (through cookies).

I exported some of Adguard's GM API and this code should be able to run on any website where the user has Adguard installed:

var settings = {
apiurl: "/adguard-ajax-api",
apiType: "/api",
exceptionRulePrefix: "@@||",
exceptionAttributes: ["elemhide", "content", "urlblock", "jsinject"],
croppedDomain: null,
urlInfo: null,
path: null,
testDomain: null
};
var utils = {
getHostWithProtocol: function () {
var hostname = document.location.hostname;
var protocol = document.location.protocol;
return protocol + "//" + hostname
},
cropDomain: function (domain) {
return domain.replace("www.", "")
},
getUrl: function (url) {
var pattern = "^(([^:/\\?#]+)?(//(([^:/\\?#]*)(?:[^/\\?#]*))?))?([^\\?#]*)(\\?([^#]*))?(#(.*))?$";
var rx = new RegExp(pattern);
var parts = rx.exec(url);
return {
host: parts[4] || "",
path: parts[7] || ""
}
}
};
function GM_xmlhttpRequest(details) {
details.url = details.url ? details.url : "";
details.data = details.data ? details.data : "";
details.headers = details.headers ? details.headers : "";
details.overrideMimeType = details.overrideMimeType ? details.overrideMimeType : "";
details.method = details.method ? details.method : "";
details.user = details.user ? details.user : "";
details.password = details.password ? details.password : "";
var encodeHeaders = function (headers) {
if (typeof headers == "object") {
var result = "";
for (header in headers) result += encodeURIComponent(header) + ":" + encodeURIComponent(headers[header]) + ",";
return result.slice(0, -1)
}
if (typeof headers == "string") return encodeURIComponent(headers);
return null
};
var url = settings.testDomain ? settings.testDomain : utils.getHostWithProtocol();
url += settings.apiurl + settings.apiType + "?type=gm-xml-http-request" + "&url=" + encodeURIComponent(details.url) + "&data=" + encodeURIComponent(details.data) + "&headers=" + encodeHeaders(details.headers) + "&method=" + encodeURIComponent(details.method) + "&overridemimetype=" + encodeURIComponent(details.overrideMimeType) +
"&user=" + encodeURIComponent(details.user) + "&password=" + encodeURIComponent(details.password);
var async = details.synchronous ? !details.synchronous : true;
var xmlhttp = new XMLHttpRequest;
xmlhttp.open("GET", url, async);
xmlhttp.onload = function () {
if (details.onload) details.onload(xmlhttp)
};
xmlhttp.onreadystatechange = function () {
if (details.onreadystatechange) details.onreadystatechange(xmlhttp)
};
xmlhttp.onerror = function () {
if (details.onerror) details.onerror(xmlhttp)
};
if (async) {
xmlhttp.timeout = details.timeout ? details.timeout : 15E3;
xmlhttp.ontimeout = function () {
if (details.ontimeout) details.ontimeout(xmlhttp)
};
xmlhttp.send();
return xmlhttp
} else {
xmlhttp.send();
return xmlhttp.responseText
}
}

 

[steve1209]

Hi New Version seems to have a bug for me in Version 5.10.1124.5747, when I look at update screen, it doesn't show last update see here

 

[streetwolf]

Hi New Version seems to have a bug for me in Version 5.10.1124.5747, when I look at update screen, it doesn't show last update see here

Same here.

 

[Boo Berry]

Confirmed!

Also the Chrome x64 issue still exists with both dev and canary builds. Hopefully 37 x64 goes to beta soon!

---------- Post added at 07:18 PM ---------- Previous post was at 06:55 PM ----------

This was posted here in regards to enhance Adguard's compatibility with YouTube Center (and other scripts); https://github.com/YePpHa/YouTubeCenter/issues/639#issuecomment-48551616

The reason the GM_xmlhttpRequest function wasn't working was because they return the xmlhttp variable, which is the result of new XMLHttpRequest(). They need to make a responseState object, which should look something like this:

var responseState = {
responseXML: "",
responseText: (xmlhttp.readyState == 4 ? xmlhttp.responseText : ""),
readyState: xmlhttp.readyState,
responseHeaders: (xmlhttp.readyState == 4 ? xmlhttp.getAllResponseHeaders() : ""),
status: (xmlhttp.readyState == 4 ? xmlhttp.status : 0),
statusText: (xmlhttp.readyState == 4 ? xmlhttp.statusText : ''),
finalUrl: (xmlhttp.readyState == 4 ? xmlhttp.finalUrl : '')
}
They should then return that object instead of the xmlhttp object.

Another thing I noticed was that they set the onload and onerror listeners to the xmlhttp object, where they need to be called from the onreadystatechange by checking the ready state of the xmlhttp object. So they should replace xmlhttp.onload = function() { ... }, xmlhttp.onreadystatechange = function(){ ... } and xmlhttp.onerror = function() { ... } with or very close to:

xmlhttp.onreadystatechange = function(){
var responseState = {
responseXML: "",
responseText: (xmlhttp.readyState == 4 ? xmlhttp.responseText : ""),
readyState: xmlhttp.readyState,
responseHeaders: (xmlhttp.readyState == 4 ? xmlhttp.getAllResponseHeaders() : ""),
status: (xmlhttp.readyState == 4 ? xmlhttp.status : 0),
statusText: (xmlhttp.readyState == 4 ? xmlhttp.statusText : ""),
finalUrl: (xmlhttp.readyState == 4 ? xmlhttp.finalUrl : "")
};
if (details["onreadystatechange"]) {
details["onreadystatechange"](responseState);
}
if (xmlhttp.readyState == 4) {
if (details["onload"] && xmlhttp.status >= 200 && xmlhttp.status < 300) {
details["onload"](responseState);
}
if (details["onerror"] && (xmlhttp.status < 200 || xmlhttp.status >= 300)) {
details["onerror"](responseState);
}
}
};
This will most likely also increase the compatibility with other userscripts too.

 

[streetwolf]

A couple of ads are not collapsing as they did before. I put a red square around them in the screenshot. Note this happens on other pages at Neowin.

Also, why would I choose to use for example the EasyList subscription over or with your English filter? Are using too many filters not a good thing to do?

Also, can't check the box for 'Show useful ads".

---------- Post added at 09:15 PM ---------- Previous post was at 08:36 PM ----------

Lots of non-ad related stuff missing at http://www.howtogeek.com/

 

[marke68]

I'm also getting the "databases not loaded" bug.

 

[Rian]

Same here

 

[Himanshu]

Was about to report the same....

 

[avatar]

Thank you for all these bugs found! We'll release a new version today, fixing cosmetic issues.

---------- Post added at 03:13 PM ---------- Previous post was at 03:04 PM ----------

@Boo Berry

Thank you for YTC dev code review! It was really helpful

---------- Post added at 03:15 PM ---------- Previous post was at 03:13 PM ----------

Also, can't check the box for 'Show useful ads".

Cannot reproduce it, can you give me more details on this issue?
UPD: ok, I've reproduced it now.

---------- Post added at 03:22 PM ---------- Previous post was at 03:15 PM ----------

Thanks for the release. Is there any way to add custom (unlisted) Adblock filter list? I would like to add one from:
http://adblocklist.org/

Not yet.

Adguard's behavior is slightly different from ABP.
So the subscriptions you see are served from our servers and transformed for better compatibility.

---------- Post added at 03:25 PM ---------- Previous post was at 03:22 PM ----------

I noticed that YouTube Center's settings (I assume local storage or something else) is being saved globally between all browsers, and not for each specific browser like it was doing not too long ago. This can be a problem as I prefer to force HTML5 in Chrome, while forcing Flash in Chromium (due to the lack of H.264 support in Chromium). Same with Firefox and Nightly, I prefer to force Flash for Firefox (until at least FF31) and HTML5 for Nightly. Is this as intended? Is it a bug or an oversight? Is it an issue with the script itself?

It is intended. I think that YTC have been using local storage before and that's why you had different settings in different browsers.
But now it sees GM_ functions implementations and uses them.

---------- Post added at 03:31 PM ---------- Previous post was at 03:25 PM ----------

@rickyoh
Thank you for the bug reports!

-since the "Adblock Warning Removal List" is part of the Easylist group of filters, wouldn't it make sense to have it appear under EasyList ?

It is developed by EasyList authors but it is not part of EasyList:
https://adblockplus.org/en/subscriptions

- shouldn't the homepage link next to the "User filter" entry in the Ad Blocker section load the user filter section of Adguard rather than Adguard's homepage ?

Good idea, we'll do it

 

[streetwolf]

In my previous post I showed a screenshot of ads that were not collapsed at neowin.com. I since 'rediscovered' that this was always the case with AG. Manually blocking these gray boxes with AG does nothing to get rid of them. However, I also run AdBlock Edge along with Element Helper. Using Element Helper I am able to collapse these boxes. Shouldn't the boxes be automatically collapsed by AG in the first place and why does your block element function not collapse them as does Element Helper?

As far as the problem I reported on the HowToGeek site, this was caused by some filter in Fanboy's Annoyance list. I'm now just using Fanboy's social list.

 

[avatar]

The widget.banner-square element on Neowin.net isn't being blocked anymore.

Adguard 5.10 now supports CSS injection rules: http://adguard.com/en/filterrules.html#cssInjection
But it ignores our "workaround" rules (#adg_start_style_inject) which we've used for neowin.

I've updated English filter, now everything should be ok.

 

[streetwolf]

Not sure if this is a bug or not. I removed the Fanboy's Annoyance List using AG's main menu. However, it still showed up on the filters list using the AG tray icon. Closing AG and restarting it cleared it up.

*** Edit
Adding a subscription doesn't make it appear using the AG tray icon.

 

[avatar]

Not sure if this is a bug or not. I removed the Fanboy's Annoyance List using AG's main menu. However, it still showed up on the filters list using the AG tray icon. Closing AG and restarting it cleared it up.

*** Edit
Adding a subscription doesn't make it appear using the AG tray icon.

Confirmed, tray icon menu works really bad now.