Compare commits

..

No commits in common. "master" and "v0.2.1" have entirely different histories.

17 changed files with 92 additions and 290 deletions

3
.github/FUNDING.yml vendored
View File

@ -1,3 +0,0 @@
ko_fi: captn3m0
liberapay: captn3m0
github: captn3m0

1
.gitignore vendored
View File

@ -9,4 +9,3 @@
/vendor/
Gemfile.lock
*.gem
.env

View File

@ -7,34 +7,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## 1.0.2 - 2024-03-07
### Changed
- Dependency updates
- Fixes for new Outline API
## 1.0.1 - 2020-04-26
### Changed
- Dependency Updates
## 1.0.0 - 2020-04-26
### Changed
- Changes `push` to `sync`.
### Added
- `sync` now actually syncs the git repository, and maintains history.
## 0.2.2 - 2019-08-14
- Adds a `push` command (See #2)
## 0.2.0 - 2019-08-12
- Adds export command (See [#1013](https://github.com/outline/outline/pull/1013) for corresponding Outline PR)

View File

@ -1,18 +1,9 @@
FROM ruby:3.3-alpine3.19
RUN apk add --no-cache git openssh-client rsync && \
echo -e "StrictHostKeyChecking no" >> /etc/ssh/ssh_config && \
mkdir /root/.ssh
FROM ruby:2.6-alpine
WORKDIR /outliner
COPY . /outliner/
RUN echo "gem: --no-ri --no-rdoc" > ~/.gemrc && \
apk add --no-cache alpine-sdk && \
gem update --system && \
gem install bundler && \
bundle install && \
apk del --no-cache alpine-sdk && \
rm ~/.gemrc
RUN gem install bundler && \
bundle install
ENTRYPOINT ["/outliner/entrypoint.sh"]

View File

@ -1,7 +1,7 @@
Copyright 2020 Abhay Rana
Copyright 2019 Abhay Rana
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@ -1,4 +1,4 @@
# Outliner [![Gem Version](https://badge.fury.io/rb/outliner.svg)](https://badge.fury.io/rb/outliner) [![Docker Cloud Build Status](https://img.shields.io/docker/cloud/build/captn3m0/outliner)](https://hub.docker.com/r/captn3m0/outliner) [![Docker Image Version (latest semver)](https://img.shields.io/docker/v/captn3m0/outliner)](https://hub.docker.com/r/captn3m0/outliner) [![Docker Image Size (latest semver)](https://img.shields.io/docker/image-size/captn3m0/outliner)](https://hub.docker.com/r/captn3m0/outliner)
# Outliner [![Gem Version](https://badge.fury.io/rb/outliner.svg)](https://badge.fury.io/rb/outliner) [![](https://images.microbadger.com/badges/version/captn3m0/outliner:v0.2.0.svg)](https://microbadger.com/images/captn3m0/outliner:v0.2.0 'Get your own version badge on microbadger.com') [![](https://images.microbadger.com/badges/version/captn3m0/outliner:latest.svg)](https://microbadger.com/images/captn3m0/outliner:latest 'Get your own version badge on microbadger.com')
A simple HTTParty based wrapper for the [Outline API](https://www.getoutline.com/developers). It also offers a one-line import option to let you migrate an existing set of Markdown files to Outline. For quickly running export/import commands, you can use the Docker Image as well.
@ -28,13 +28,6 @@ require 'outliner'
client = Outliner.new('https://knowledge.example.com')
pp client.auth_info
pp client.collections_list(offset: 0, limit: 10)
# This works around a 302 redirect bug in httparty
begin
r = @client.fileOperations__redirect({id: FILE_OPERATION_ID}, format: nil, no_follow: true)
rescue HTTParty::RedirectionTooDeep => e
# Download this using response = HTTParty.get e.response.header['location'] if needed
pp e.response.header['location']
end
```
### Import
@ -67,16 +60,11 @@ outliner-export "$DESTINATION_DIRECTORY"
You can use the pre-built docker image to run the above commands as well. See the following commands for examples:
### Setup
Copy the `env.sample` file to `.env` and update the values there.
### Export
Downloads all collections from Outline, and exports them as nested markdown files inside the given directory (`/data` inside the container, mount it accordingly.)
```bash
docker run --env-file .env
docker run --env OUTLINE_BASE_URI="https://kb.example.com" \
--env OUTLINE_TOKEN="PUT YOUR TOKEN HERE" \
--volume /tmp:/data \
captn3m0/outliner \
export \
@ -85,33 +73,18 @@ docker run --env-file .env
### Import
Imports all markdown documents in a directory to a named Collection on outline. Creates the collection if it doesn't exist.
```bash
docker run --env-file .env
docker run --env OUTLINE_BASE_URI="https://kb.example.com" \
--env OUTLINE_TOKEN="PUT YOUR TOKEN HERE" \
--volume /path/to/wiki:/data \
captn3m0/outliner \
import /data "Archive"
```
### Sync
Does a export from Outline, and pushes the corresponding result to the Git repository. Currenly does a force-push to the repository. Use with care.
Note: Sync is currently only available as a Docker Command
```bash
docker run --env-file .env
--volume /etc/ssh/private.key:/root/.ssh/id_rsa
captn3m0/outliner \
sync
import "/data" "Archive"
```
#### Limitations
- [import] Images are currently not imported. Host them externally for this to work.
- [import] Only `.md` files are currently supported
- [docker] `StrictHostKeyChecking` is currently disabled for `push`, please only run this in trusted networks.
- Images are currently not imported. Host them externally for this to work.
- Only `.md` files are currently supported
## Development

View File

@ -1,77 +1,19 @@
#!/bin/sh
set -eu
if [ $# -eq 0 ]; then
echo "Please run with outliner [export|import|sync] arguments"
echo "Please run with outliner [export|import] arguments"
exit
fi
setup_git() {
if [ -f "$HOME/.ssh/id_rsa" ]; then
# This is required because Kubernetes secret mounts can't
# have file permissions set
chmod 0400 "$HOME/.ssh/id_rsa"
if [ ! -d "$HOME/.ssh/id_rsa.pub" ]; then
ssh-keygen -y -f "$HOME/.ssh/id_rsa" > "$HOME/.ssh/id_rsa.pub"
fi
echo "[+] Using SSH key for git pushes"
else
echo "[E] Git credentials not available, quitting"
exit 1
fi
eval $(ssh-agent)
ssh-add "$HOME/.ssh/id_rsa"
}
update_git_config() {
EMAIL=${GIT_EMAIL:-outliner@example.invalid}
git config --global user.email "$EMAIL"
git config --global user.name "Outliner Backup"
git remote add origin "$GIT_REMOTE_URL"
}
case $1 in
export)
shift
bundle exec outliner-export "$@"
bundle exec outliner-export $@
;;
import)
shift
bundle exec outliner-import "$@"
;;
sync)
BRANCH=${GIT_BRANCH:-master}
old_git_dir=$(mktemp -d)
fresh_export_dir=$(mktemp -d)
if [ -z "$GIT_REMOTE_URL" ]; then
echo "[E] GIT_REMOTE_URL not set"
exit 1
else
git clone --branch "$BRANCH" "$GIT_REMOTE_URL" "$old_git_dir"
setup_git
update_git_config
echo "[+] Exporting data from Outline"
bundle exec outliner-export "$fresh_export_dir"
echo "[+] Resetting git repository"
cd "$old_git_dir"
# We update so that git forgets all files
git ls-files -z |xargs -n1 -0 git rm
# Then we copy across the files from the new export
cd "$fresh_export_dir"
echo "[+] Updating git repository"
rsync -av . "$old_git_dir"
cd "$old_git_dir"
echo "[+] Committing to git"
git add .
git commit --message "Backup: $(date)" > /dev/null
git status
echo "[+] Pushing to git remote"
git push origin "HEAD:$BRANCH"
echo "[+] Cleaning up"
rm -rf "$old_git_dir"
rm -rf "$fresh_export_dir"
fi
bundle exec outliner-import $@
break
;;
*)
echo "Invalid command, please check README"

View File

@ -1,4 +0,0 @@
OUTLINE_BASE_URI=https://outline.example.com
GIT_BRANCH=master
GIT_REMOTE_URL=git@example.com:org/outline.backup.git
OUTLINE_TOKEN=

View File

@ -1,15 +1,16 @@
#!/usr/bin/env ruby
# frozen_string_literal: true
require 'bundler/setup'
require 'outliner'
require "bundler/setup"
require "outliner"
require 'tempfile'
def validate
raise 'Missing arguments' if ARGV.size != 1
raise 'Invalid directory' unless Dir.exist?(ARGV[0])
raise 'OUTLINE_BASE_URI not set' unless ENV.key?('OUTLINE_BASE_URI')
raise 'OUTLINE_TOKEN not set' unless ENV.key?('OUTLINE_TOKEN')
unless (ARGV.size == 1) and Dir.exists?(ARGV[0]) and ENV.key?('OUTLINE_BASE_URI') and ENV.key?('OUTLINE_TOKEN')
puts "[E] Please call as `outliner-export directory`"
puts "[E] Please export OUTLINE_BASE_URI and OUTLINE_TOKEN environment variables"
puts "[E] OUTLINE_BASE_URI should not include /api"
exit 1
end
end
# Run validations
@ -20,30 +21,13 @@ local_directory = ARGV[0]
CLIENT = Outliner::Client.new ENV['OUTLINE_BASE_URI']
# Download the complete zip
response = CLIENT.collections__export_all(format: "outline-markdown")
response = CLIENT.collections_exportAll(download: true)
raise 'Failed to trigger export_all action' if not response['success']
file_operation_id = response['data']['fileOperation']['id']
fop_info_response = nil
i = 0
loop do
i += 1
raise 'Timed out waiting for the file export operation to complete' if i > 20
sleep(2*i)
fop_info_response = CLIENT.fileOperations__info(id: file_operation_id)
raise 'Failed to query export file operation info' if not fop_info_response['ok']
break if fop_info_response['data']['state'] == 'complete'
end
begin
fop_redirect_response = CLIENT.fileOperations__redirect({id: file_operation_id}, {no_follow: true})
rescue HTTParty::RedirectionTooDeep => e
response = HTTParty.get e.response.header['location']
file = Tempfile.new('download.zip')
File.open(file.path, 'w') { |f| f.write(response.body) }
`unzip -o "#{file.path}" -d "#{local_directory}"`
file.unlink
end
# Extract it to a tempfle
file = Tempfile.new('download.zip')
File.open(file.path, 'w') { |file| file.write(response.body) }
`unzip -o "#{file.path}" -d "#{local_directory}"`
# Delete tempfile
file.unlink

View File

@ -1,48 +1,45 @@
#!/usr/bin/env ruby
# frozen_string_literal: true
require 'bundler/setup'
require 'outliner'
require "bundler/setup"
require "outliner"
def validate
raise 'Missing arguments' if ARGV.size != 2
raise 'Invalid directory' unless Dir.exist?(ARGV[0])
raise 'Invalid collection' unless ARGV[1].match(/\w+/)
raise 'OUTLINE_BASE_URI not set' unless ENV.key?('OUTLINE_BASE_URI')
raise 'OUTLINE_TOKEN not set' unless ENV.key?('OUTLINE_TOKEN')
unless (ARGV.size == 2) and Dir.exists?(ARGV[0]) and ARGV[1].match(/\w+/) and ENV.key?('OUTLINE_BASE_URI') and ENV.key?('OUTLINE_TOKEN')
puts "[E] Please call as `outliner-import local_directory remote_collection_name`"
puts "[E] Please export OUTLINE_BASE_URI and OUTLINE_TOKEN environment variables"
puts "[E] OUTLINE_BASE_URI should not include /api"
exit 1
end
end
def create_documents_recursively(directory, collection_id, parent_document_id = nil)
def create_documents_recursively(directory, collection_id, parent_document_id=nil)
cwd = Dir.pwd
Dir.chdir directory
# Create all documents for this directory
Dir['*.md'].each do |file|
Dir["*.md"].each do |file|
params = {
title: file[0...-3],
text: file[0...-3] +
"\n" +
File.read(file) +
"\n\n---\nImported at #{Time.now}",
text: file[0...-3] + "\n" + File.read(file) + "\n\n---\nImported at #{Time.now}",
collectionId: collection_id,
publish: true
}
params[:parentDocumentId] = parent_document_id if parent_document_id
CLIENT.documents__create(params)
CLIENT.documents_create(params)
puts "[-] #{file}"
end
# Create child documents for each sub-directory
Dir.glob('*').select { |f| File.directory? f }.each do |dir|
Dir.glob('*').select {|f| File.directory? f}.each do |dir|
puts "[-] #{dir}"
params = {
title: dir,
text: dir + "\nImported at #{Time.now}",
text: dir +"\nImported at #{Time.now}",
collectionId: collection_id,
publish: true,
parentDocumentId: parent_document_id
}
response = CLIENT.documents__create(params)
response = CLIENT.documents_create(params)
create_documents_recursively(dir, collection_id, response['data']['id'])
end
Dir.chdir cwd
@ -61,11 +58,12 @@ root_collection_id = CLIENT.find_or_create_collection(remote_collection_name)
begin
create_documents_recursively(local_directory, root_collection_id)
puts '[S] Import successful'
rescue StandardError? => e
puts "[S] Import successful"
rescue Exception => e
# If we fail, print an error, and delete the collection
puts "[E] Import failed with error: #{e.message}"
CLIENT.collections__delete(id: root_collection_id)
puts '[E] Deleted collection, please report the issue or retry'
CLIENT.collections_delete(id: root_collection_id)
puts "[E] Deleted collection, please report the issue or retry"
exit 1
end

View File

@ -11,28 +11,27 @@ module Outliner
end
def find_or_create_collection(name)
collections = self.collections__list(limit: 100)['data']
collections = self.collections_list(limit: 100)['data']
collections.filter!{|c|c['name'] == name}
if collections.size >= 1
collections[0]['id']
else
self.collections__create(name: name, description: 'Imported Collection')['data']['id']
self.collections_create(name: name, description: 'Imported Collection')['data']['id']
end
end
def method_missing(method_name, params = {}, options = {})
method_name = "/#{method_name.to_s.sub('__', '.')}"
def method_missing(method_name, params = {})
method_name = '/' + method_name.to_s.sub('_', '.')
body = {token: @token}.merge(params).to_json
options = {
body: params.to_json,
body: body,
headers: {
'Accept' => 'application/json',
'Content-Type' => 'application/json',
'User-Agent' => "Outliner/#{Outliner::VERSION}",
'Authorization' => "Bearer #{@token}"
'Accept'=>'application/json',
'Content-Type': 'application/json',
'User-Agent': "Outliner/#{Outliner::VERSION}"
},
format: :json,
}.merge!(options)
format: :json
}
self.class.post(method_name, options)
end

13
lib/outliner/helper.rb Normal file
View File

@ -0,0 +1,13 @@
module Outliner
class Helper
def find_or_create_collection(client, name)
collections = client.collections_list(limit: 100)['data']
collections.filter!{|c|c['name'] == name}
if collections.size >= 1
collections[0]['id']
else
client.collections_create(name: name, description: 'Imported Collection')['data']['id']
end
end
end
end

View File

@ -1,3 +1,3 @@
module Outliner
VERSION = "1.0.2"
VERSION = "0.2.1"
end

View File

@ -7,7 +7,7 @@ Gem::Specification.new do |spec|
spec.name = "outliner"
spec.version = Outliner::VERSION
spec.authors = ["Nemo"]
spec.email = ["outliner@captnemo.in"]
spec.email = ["rubygem.outliner@captnemo.in"]
spec.licenses = ["MIT"]
spec.summary = "A simple HTTParty based client for outline knowledge base."
@ -23,11 +23,10 @@ Gem::Specification.new do |spec|
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
spec.require_paths = ["lib"]
spec.add_dependency "httparty", "~> 0.21"
spec.add_dependency "json", "~> 2.7"
spec.add_dependency "httparty", "~> 0.17"
spec.add_development_dependency "bundler", "~> 2.1"
spec.add_dependency "rake", ">= 13.1"
spec.add_development_dependency "webmock", "~> 3.23"
spec.add_development_dependency "minitest", "~> 5.22"
spec.add_development_dependency "bundler", "~> 2.0"
spec.add_development_dependency "rake", "~> 10.0"
spec.add_development_dependency "webmock", "~> 3.6.0"
spec.add_development_dependency "minitest", "~> 5.8.4"
end

View File

@ -1,31 +0,0 @@
{
"success": true,
"data": {
"fileOperation": {
"id": "08d5db26-bf43-4ec9-ac62-8769fd828e94",
"type": "export",
"format": "outline-markdown",
"name": "Acme-export.zip",
"state": "creating",
"error": null,
"size": "0",
"collectionId": null,
"user": {
"id": "817fb131-4a9b-4981-9002-38c2503adc3e",
"name": "Acme Admin",
"avatarUrl": "https://fake-avatar-url.com",
"color": "#2BC2FF",
"isAdmin": true,
"isSuspended": false,
"isViewer": false,
"createdAt": "2024-03-07T04:03:45.204Z",
"updatedAt": "2024-03-07T06:51:26.023Z",
"lastActiveAt": "2024-03-07T06:51:26.023Z"
},
"createdAt": "2024-03-07T06:51:26.031Z",
"updatedAt": "2024-03-07T06:51:26.031Z"
}
},
"status": 200,
"ok": true
}

View File

@ -1 +0,0 @@
Redirecting to https://fake.s3-accelerate.amazonaws.com/uploads/3e11b7f9-f1c0-44d0-a21b-4d6e0561e9c9/a5b6985a-cff6-4d03-be60-20c517bee63e/Acme-export.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=XXXXXXXXXXXXXXXXXXXX%2F20240307%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240307T055812Z&X-Amz-Expires=60&X-Amz-Signature=ff759b27ddfd5c7401c1715411a8ceba886f9f462c9b52fc0c4a5906e99ecd22&X-Amz-SignedHeaders=host&response-content-disposition=attachment.

View File

@ -6,7 +6,6 @@ require 'json'
class ClientTest < Minitest::Test
TOKEN = "c4302eFAKE_TOKEN9b6e27bccb7"
BASE_URI='https://kb.example.com'
FILE_OPERATION_ID = "08d5db26-bf43-4ec9-ac62-8769fd828e94"
def setup
ENV['OUTLINE_TOKEN'] = TOKEN
@client = Outliner::Client.new BASE_URI
@ -17,36 +16,9 @@ class ClientTest < Minitest::Test
end
def test_auth_info_api
mock('auth.info')
r = @client.auth__info
assert_equal "https://kb.example.com", r['data']['team']['url']
end
def test_export
mock('collections.export_all')
r = @client.collections__export_all
assert_equal FILE_OPERATION_ID, r['data']['fileOperation']['id']
assert_equal 200, r['status']
assert_equal true, r['ok']
end
def test_retrieve_file_operation
mock("fileOperations.redirect", {
id: FILE_OPERATION_ID
}, {
"X-Download-Options" => "noopen",
"X-Content-Type-Options" => "nosniff",
"Content-Type" => "text/plain; charset=utf-8",
"Content-Length" => "459",
"Location" => "https://s3.example.com/#{FILE_OPERATION_ID}"
}, 302)
begin
r = @client.fileOperations__redirect({id: FILE_OPERATION_ID}, format: nil, no_follow: true)
rescue HTTParty::RedirectionTooDeep => e
assert_equal "302", e.response.code
assert_equal "https://s3.example.com/#{FILE_OPERATION_ID}", e.response.header['location']
end
mock('auth.info', 'auth.info.200')
auth_info = @client.auth_info
assert_equal "https://kb.example.com", auth_info['data']['team']['url']
end
private
@ -55,16 +27,15 @@ class ClientTest < Minitest::Test
File.read "test/fixtures/#{file}.json"
end
def mock(method_name, params = {}, response_headers = {}, status = 200)
def mock(method_name, fixture_file, params = {})
stub_request(:post, BASE_URI + "/api/" + method_name)
.with(
body: params.to_json,
body: params.merge({token: TOKEN}).to_json,
headers: {
'Accept'=>'application/json',
'User-Agent'=>"Outliner/#{Outliner::VERSION}",
'Content-Type'=> 'application/json',
"Authorization"=> "Bearer #{TOKEN}"
'Content-Type'=> 'application/json'
}
).to_return(body: read_fixture(method_name + ".#{status}"), headers: response_headers, status: 302)
).to_return(body: read_fixture(fixture_file))
end
end
end