Django haystack with AWS open search (elastic search compatible)

Im trying to use AWS open search with Django Haystacks, which is fully compatible (Amazon have suggested). However when I try launch "./manage.py rebuild_index" when pointing a connection at the platform I am met with the error

Traceback (most recent call last):
  File "/data/app/myapp/./manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 402, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 448, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.10/site-packages/haystack/management/commands/rebuild_index.py", line 64, in handle
    call_command("clear_index", **clear_options)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 198, in call_command
    return command.execute(*args, **defaults)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 448, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.10/site-packages/haystack/management/commands/clear_index.py", line 64, in handle
    backend.clear(commit=self.commit)
  File "/usr/local/lib/python3.10/site-packages/haystack/backends/elasticsearch7_backend.py", line 117, in clear
    self.conn.indices.delete(index=self.index_name, ignore=404)
  File "/usr/local/lib/python3.10/site-packages/elasticsearch/client/utils.py", line 347, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/elasticsearch/client/indices.py", line 334, in delete
    return self.transport.perform_request(
  File "/usr/local/lib/python3.10/site-packages/elasticsearch/transport.py", line 421, in perform_request
    _ProductChecker.raise_error(self._verified_elasticsearch)
  File "/usr/local/lib/python3.10/site-packages/elasticsearch/transport.py", line 638, in raise_error
    raise UnsupportedProductError(message)
elasticsearch.exceptions.UnsupportedProductError: The client noticed that the server is not a supported distribution of Elasticsearch

when we've dug into this by looking in "site-packages/elasticsearch/transport.py" and the differences between the amazon headers and an elastic search header the only differences between the two is that amazon openserach "build_flavour" in it. Build flavour is used in the check_product function from transport.py

@classmethod
    def check_product(cls, headers, response):
        # type: (dict[str, str], dict[str, str]) -> int
        """Verifies that the server we're talking to is Elasticsearch.
        Does this by checking HTTP headers and the deserialized
        response to the 'info' API. Returns one of the states above.
        """
    try:
        version = response.get("version", {})
        version_number = tuple(
            int(x) if x is not None else 999
            for x in re.search(
                r"^([0-9]+)\.([0-9]+)(?:\.([0-9]+))?", version["number"]
            ).groups()
        )
    except (KeyError, TypeError, ValueError, AttributeError):
        # No valid 'version.number' field, effectively 0.0.0
        version = {}
        version_number = (0, 0, 0)

    # Check all of the fields and headers for missing/valid values.
    try:
        bad_tagline = response.get("tagline", None) != "You Know, for Search"
        bad_build_flavor = version.get("build_flavor", None) != "default"
        bad_product_header = (
            headers.get("x-elastic-product", None) != "Elasticsearch"
        )
    except (AttributeError, TypeError):
        bad_tagline = True
        bad_build_flavor = True
        bad_product_header = True

    # 7.0-7.13 and there's a bad 'tagline' or unsupported 'build_flavor'
    if (7, 0, 0) <= version_number < (7, 14, 0):
        if bad_tagline:
            return cls.UNSUPPORTED_PRODUCT
        elif bad_build_flavor:
            return cls.UNSUPPORTED_DISTRIBUTION

    elif (
        # No version or version less than 6.x
        version_number < (6, 0, 0)
        # 6.x and there's a bad 'tagline'
        or ((6, 0, 0) <= version_number < (7, 0, 0) and bad_tagline)
        # 7.14+ and there's a bad 'X-Elastic-Product' HTTP header
        or ((7, 14, 0) <= version_number and bad_product_header)
    ):
        return cls.UNSUPPORTED_PRODUCT

    return True

I commented out the build flavour checks in this function, and then the index built successfully! So I went to post it as an issue on the haystacks GitHub and I was going to fork it so I could remove those lines, but the file "elasticsearch/transport.py" and the code "def check_product" doesn't exist in that repo (when I search the repo via GitHub), its pulled from somewhere else, and I can't find it online anywhere.

hopefully someone can point me in the right direction?

Thanks

Back to Top