Rest API Doc out of date

[pennal]

Hi,

I am trying to implement a simple import/export script to integrate Tolgee in a CI/CD workflow and I am constantly hitting errors with the API. The current documentation (especially around the fields and explanations of the parameters) is not consistent to what the code is doing behind the scenes.

I have currently started to sniff out the requests made by the UI since those seem to work, but this is not sustainable in the long run. An example is the <host>/v2/projects endpoint which is not in the Docs, but is heavily used by the UI.

Also, would you consider allowing the export of the OpenAPI spec so a library (and the different DTOs) could be generated on the fly?

[JanCizmar]

Hey! The full OpenAPI spec is described here. You can generate API spec from the json definition as you wish.

If there's anything inconsistent or you're hitting errors, please specify such issues. 🤘🏻

[JanCizmar]

But Yes! It has to be reviewed and fixed.

[pennal]

Hi @JanCizmar, thanks for your answer. Thanks to the Swagger doc I have been able to better unserstand how the rest API looks. But unfortunately there are a couple of issues:

Swagger codegen creates unusable client

I tried generating the client with swagger-codegen, but due to the auth method used the client, once instantiated is unusable as it fails with a no auth provided error of some sorts. I generated the lib with:

java -jar swagger-codegen.jar generate -i swagger.json -l java -o tolgee-client-library-java -DhideGenerationTimestamp=true

The library is generated just fine, and all assets seem to be created correctly, but when I instantiate the client with

ApiClient apiClient = new ApiClient();
apiClient.setApiKey(System.getenv("API_KEY"));
apiClient.setBasePath(System.getenv("URL"));

LanguagesApi languagesApi = new LanguagesApi(apiClient);
PagedModelLanguageModel languageModel = languagesApi.getAll6(1, 50, null, null, null);
List<LanguageModel> languageModels = languageModel.getEmbedded().getLanguages();
languageModels.stream().peek(System.out::println);

But the following error is thrown:

Exception in thread "main" java.lang.RuntimeException: No API key authentication configured!
	at com.my-thing.tolgee.library.ApiClient.setApiKey(ApiClient.java:309)
	at com.my-thing.Main.main(Main.java:31)

Line 31 is where I set the Api Key, i.e. apiClient.setApiKey(System.getenv("API_KEY"));. As a workaround for now, I have created my own custom client, but for future proofing I would prefer to use the generated client. I believe this is not an issue with the swagger codegen (I have used multiple versions, and the latest as of 20.12.22).

Exporting translations works only when the api key is NOT provided

The route specified in the spec fails with an error (don't have the specific one, sorry), but ONLY when the api key is provided. I used the same route that the UI calls (/v2/projects/{projectId}/export) without the API key and all works fine, the bundle is downloaded without major issues. The only downside of this approach is when using user keys and not project keys, but for now I am using a project key therefore only one project is returned and is easy to find the id.

Sorry if I don't have too many details, if needed let me know and I can provide them

[JanCizmar]

Hey! The No API key authentication configured error message is definitely not thrown by Tolgee. It's probably issue with your approach of providing the API key. Try to provide the API key via setting custom header to client instead using the setApiKey method.

Anyway, we don't guarantee that our swagger docs work with specific codegen tools, so this is not an Tolgee issue.

[pennal]

Yes, you are right that this error is thrown by Swagger (and the generated client), but it is due to how the swaggerfile has been specified. I have used this generator on multiple other projects and it works without any issues, so my suspects are with the double auth implemented in Tolgee (and the fact that Swagger can't really understand it).

I believe that once the old method (with URL params) is taken out, the generator will create the correct client, and therefore this issue will be irrelevant.

For now, I'll keep using my own client, but will reconsider once this gets fixed by either side.

[jdimeo]

@pennal I had to make tweaks to the spec to get the auth working the way I wanted. For example, adding an API key globally for the internal API:

Also, I've noticed that some of the secondary HTTP verbs of an endpoint can be missing parameters:

Over time, the swagger spec has been needing less and less tweaks to get it to work with (in my case) OpenAPI generator, and I just use Git to track any modifications so that I can replay them when I download the spec again. It still saves a ton of time since it's 95% of the way there

[pennal]

@jdimeo thanks for your input. As you said, the spec is 95% there but for me the auth part is blocking me and preventing me from using it properly.

Hopefully this will be fixed in a future release!

[jdimeo]

I just updated the spec just now and a) it was still missing the security I needed and b) there were 4 missing parameters. (my line numbers on this patch are a bit off b/c of added security lines at the top)

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.

[jdimeo]

I just pulled the latest API spec and many parameters are no longer required. This is a problem for "path" parameters /my/{a}/path because OpenAPI spec validation fails if parameter a is not required. So I had to revert all those changes.

Also, TranslationViewModel mode is missing state OUTDATED right?

[jdimeo]

More generally, can the API specifications identify which version of Tolgee they relate to? they have always said v1.0 and so it makes it hard to track which specific version of the spec should work with which specific endpoints that have been released.

[JanCizmar]

Well, that probably happened, because we updated Spring and Spring OpenApi version. Can you please share which endpoints exactly contain these errors? Also, what do you mean by "OpenAPI spec validation fails"? Thanks!

[jdimeo]

Here's the exact output from the OpenAPI generator (which code generates API clients in many languages from a spec).

org.openapitools.codegen.SpecValidationException: There were issues with the specification. The option can be disabled via validateSpec (Maven/Gradle) or --skip-validate-spec (CLI).
 | Error count: 5, Warning count: 2
Errors:
        -paths.For path parameter languageId the required value should be true
        -paths.'/v2/projects/{projectId}/translations/{translationId}/comments/{commentId}/set-state/{state}'. Declared path parameter translationId needs to be defined as a path parameter in path or operation level
        -paths.'/v2/projects/{projectId}/translations/{translationId}/comments/{commentId}'. Declared path parameter translationId needs to be defined as a path parameter in path or operation level
        -paths.For path parameter code the required value should be true
        -paths.For path parameter projectId the required value should be true
Warnings:
        -paths.For path parameter languageId the required value should be true
        -paths.'/v2/projects/{projectId}/translations/{translationId}/comments/{commentId}/set-state/{state}'. Declared path parameter translationId needs to be defined as a path parameter in path or operation level
        -paths.'/v2/projects/{projectId}/translations/{translationId}/comments/{commentId}'. Declared path parameter translationId needs to be defined as a path parameter in path or operation level
        -paths.For path parameter code the required value should be true
        -paths.For path parameter projectId the required value should be true

Here's the command: https://openapi-generator.tech/docs/usage/#validate

[jdimeo]

tolgee-sdk-07-48-54.patch
Here's the full patch between the latest internal spec at https://app.tolgee.io/v3/api-docs/V2%20Internal%20-%20for%20Tolgee%20Web%20application vs. what I'm using to code generate a client (you can see I'm adding security, missing parameters, and leaving the previous required settings).

[JanCizmar]

Thanks! It's some sort of bug that the path is missing for some of the errors and warnings?

[jdimeo]

Yes, if the spec validation fails, the client doesn't generate the code. So I have to manually keep patching the OpenAPI spec so that the client will generate so I can continue using the API (which is how I'm interacting with all Tolgee APIs so I get a full client in any language we need).

[JanCizmar]

I am trying to play with that locally, but it seems like the output is incomplete:

        -paths.<Path is not specified here>For path parameter languageId the required value should be true

[jdimeo]

Huh strange that the path is not specified. I use it exclusively as the Maven/Gradle plugin. Here's my config FWIW

<plugin>
	<groupId>org.openapitools</groupId>
	<artifactId>openapi-generator-maven-plugin</artifactId>
	<version>6.5.0</version>
	<configuration>
		<generateApiTests>false</generateApiTests>
		<generateModelTests>false</generateModelTests>
		<generatorName>java</generatorName>
		<library>jersey3</library>
		<apiPackage>io.tolgee.api</apiPackage>
		<modelPackage>io.tolgee.model</modelPackage>
		<addCompileSourceRoot>true</addCompileSourceRoot>
		<globalProperties>
			<skipFormModel>false</skipFormModel>
		</globalProperties>
		<configOptions>
			<title>${project.name}</title>
			<sourceFolder>.</sourceFolder>
			<implFolder>.</implFolder>
			<dateLibrary>java8</dateLibrary>
			<serializationLibrary>jackson</serializationLibrary>
			<java8>true</java8>
		</configOptions>
	</configuration>
	<executions>
		<execution>
			<id>main-api</id>
			<goals>
				<goal>generate</goal>
			</goals>
			<configuration>
				<inputSpec>${spec-dir}/internal-v2.json</inputSpec>
			</configuration>
		</execution>
	</executions>
</plugin>

Note that my PR was accepted and will be released in 6.6.0 which was required to do avatar image uploads to Tolgee

[JanCizmar]

Well it looks like the API spec have got broken with this update of SpringDoc OpenApi: #1629

So I'll try to update it to the 1.7.0 or 2.1.0, and hopefully fix it.

[JanCizmar]

So updating to springdoc-openapi to 1.7.0 helped, which I did in #1679, so after it's released your problem will be partly solved.

• Fix errors from last spring update
• FIx openapi-generator warnings
• Fix authentication headers to be present (project api key, personal access token, jwt token)

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[jdimeo]

This is not stale. This past update only had 3 parameter fixes, which is great considering how much changed!
(still missing the security schemes)

[JanCizmar]

@Martin005 is working on that.

[jdimeo]

Another spec tweak I just had to make. Some endpoints, like exporting translations to files, returns application/zip but the OpenAPI spec just says a type of object. Switching StreamingResponseBody to binary made the generator realize it was a file that gets downloaded to a temp directory and now I can work with the data. Leaving it as object throws a "message body reader not available for application/zip" error.

[JanCizmar]

@jdimeo Can you please create a new issue for this? I am closing this one, because the initial issues are resolved now.