How to dynamically generate and execute unit tests for multiple programming languages as strings?

I am working on a system where solutions to problems are validated using predefined unit test cases. The system supports multiple programming languages, and the generated test cases need to be converted into strings for execution by an API.

The Problem: To test submitted solutions, I need to dynamically generate unit test cases for each programming language. The generated tests should:

Include predefined base test cases for each problem (e.g., validating a palindrome function).

Allow dynamic addition of custom test cases (e.g., user-defined edge cases). The generated tests must then be converted to a string and sent to Judge0 for execution.

Current Approach: For Python:

I use unittest to define base test cases in a file (e.g., test_palindrome.py). Custom test cases are dynamically added as new test methods. Both base and custom test cases are combined into a single string.

The following code will already be existing with the is_palindrome.py file but new custom test cases will need to be added on.

import unittest   

class TestPalindrome(unittest.TestCase):  
    def test_base_cases(self):  
        self.assertTrue(is_palindrome("radar"))  
        self.assertFalse(is_palindrome("hello"))

for example, for the given the user input:

[
  { "input": "'level'", "expected_output": "True" },
  { "input": "'world'", "expected_output": "False" }
]

I would need to generate a string of these test cases and add them on to the existing tests once the is_palindrome.py file is read.

    def test_custom_0(self):  
        self.assertEqual(is_palindrome('level'), True)  
    def test_custom_1(self):  
        self.assertEqual(is_palindrome('world'), False)

For other languages like Java or JavaScript, I use their respective testing frameworks (e.g., JUnit, Jest) and follow a similar approach to dynamically generate test cases as strings. I know the string generation will be a pain to do, the languages I need to cover would be the following -

  • C
  • C++
  • Python 3
  • Java
  • JavaScript
  • TypeScript
  • C#
  • Ruby
  • PHP
  • Go
  • Rust

My Questions:

Is this approach of generating test cases dynamically and converting them to strings efficient for multiple languages?

Would creating a system to genenrate test cases as string for different languages be better approach?

Would maintaining predefined test files for each problem/language and appending custom test cases dynamically before converting to a string be a better approach?

How can I handle user-defined test cases securely and efficiently while ensuring the string generation process is not a bottleneck?

Is there an easy way to generate the test cases for different languages using python?

Any advice, best practices, or examples for implementing such a system would be greatly appreciated. Thank you!

Вернуться на верх