Home/ Questions/Q 9271103
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-18T15:32:08+00:00

I’m writing a proxy server. It applies different rules to websites that match in

  • 0

I’m writing a proxy server. It applies different rules to websites that match in lists. For example, we can block List A and use another proxy to fetch content for List B.

For example, List A:

.google.com
blogger.com
sourceforge.net
ytimg.com
http://media-cache-*.pinterest.com/*
images-amazon.com
*.amazonaws.com
twitter.com
fbcdn.net
google-analytics.com
staticflickr.com

List B:

ytimg.com
youtube.com

Currently, the match function is:

struct proxy_t *
match_list(char *url) {
  // 2KB line should be enough
  char buf[2048];
  int pos = 0, size;

  struct acllist *al = config->acl_h;
  struct acl *node = al->data; 

  while (node != NULL) { // iterate list
    pos = 0; // position in list file

    size = strlen(node->data); // node->data holds a URL list

    while (1) { // iterate each line in list

      readline(buf, node->data, &pos, size);

      if (buf[0] == 0) break;

      if (strcasestr(url, buf) != NULL 
      || !fnmatch(buf, url, FNM_CASEFOLD)) {

          return node->proxy;
      }
    }
    node = node->next;
  }


  printf("Not Matched\n");

  return config->default_proxy;
}

That is, iterate the two list files, read line by line, use strcasestr and fnmatch to match a single URL.

It works fine. But if the lists get larger and more, say 10,000 lines per list and 5 lists, I suppose it won’t be an efficient solution since it is an O(N) algorithm.

I’m thinking about adding a hit counter to each match line. By ordering the match lines it may reduce the average search length. Like this:

.google.com|150
blogger.com|76
sourceforge.net|43
ytimg.com|22

Is there any other ideas on it?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    There are two ways you could go to improve performance.

    1

    First way is order the URL lists in some way and therefore you can optimize searching in it.
    Quicksort is fastest algorithm out there.

    Bubble sort is easier to implement.

    Then you can use binary search to search in the list.
    Binary search has logarithmic performance while your loop has linear, therefore it will be significantly faster on large lists.

    2

    If your lists of URLs are static, you can use special tool called flex, which enables you to parse the string just by reading it.

    This also means, then when some of your URL lists is updated, you have to write new code for parsing or create code generator.

    This is much more effective way of parsing, then any kind of sorting, because it only need N steps, when N is the length of URL you are parsing, therefore it doesn’t matter how long your list is, as long as you can write correct scanner for inputs.

    • 0