반응형
Gitrob
Gitrob 이란?
Gitrob은 Public Repositories인 Github에 Push 된 잠재적으로 민감한 파일을 찾는 데 도움이 되는 도구임.
Gitrob은 Github에 Upload 되어있는 프로젝트를 Clone 받아서
잠재적으로 민감한 파일, Commit history, 그리고 Gitrob이 가지고 있는 시그니쳐와 일치되는 파일을 찾아내서 보고해줌
검색 결과는 Web interface를 통해 제공되므로 쉽게 검색하고 분석 할 수 있음.
잠재적으로 민감함 파일
ID, Password, 인증서 등 개인적인 정보를 가지고 있는 파일 (Secret file)
Gitrob 시그니쳐란?
잠재적으로 민감한 파일들을 찾아내기 위해서는
대략적인 Scan 대상 파일 또는 문자열 목록이 필요함
이러한 목록을 모아놓은것을 Signature 라고함.
파일 확장자, 파일 내부 내용에 포함되어있으면 안되는 문자열 등이 저장되어있음.
[참고용 Gitrob 시스니쳐 ]
더보기
var Signatures = []Signature{
SimpleSignature{
part: PartExtension,
match: ".pem",
description: "Potential cryptographic private key",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".log",
description: "Log file",
comment: "Log files can contain secret HTTP endpoints, session IDs, API keys and othe
},
SimpleSignature{
part: PartExtension,
match: ".pkcs12",
description: "Potential cryptographic key bundle",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".p12",
description: "Potential cryptographic key bundle",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".pfx",
description: "Potential cryptographic key bundle",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".asc",
description: "Potential cryptographic key bundle",
comment: "",
},
SimpleSignature{
part: PartFilename,
match: "otr.private_key",
description: "Pidgin OTR private key",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".ovpn",
description: "OpenVPN client configuration file",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".cscfg",
description: "Azure service configuration schema file",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".rdp",
description: "Remote Desktop connection file",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".mdf",
description: "Microsoft SQL database file",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".sdf",
description: "Microsoft SQL server compact database file",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".sqlite",
description: "SQLite database file",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".bek",
description: "Microsoft BitLocker recovery key file",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".tpm",
description: "Microsoft BitLocker Trusted Platform Module password file",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".fve",
description: "Windows BitLocker full volume encrypted data file",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".jks",
description: "Java keystore file",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".psafe3",
description: "Password Safe database file",
comment: "",
},
SimpleSignature{
part: PartFilename,
match: "secret_token.rb",
description: "Ruby On Rails secret token configuration file",
comment: "If the Rails secret token is known, it can allow for remote code execution
},
SimpleSignature{
part: PartFilename,
match: "carrierwave.rb",
description: "Carrierwave configuration file",
comment: "Can contain credentials for cloud storage systems such as Amazon S3 and Goo
},
SimpleSignature{
part: PartFilename,
match: "database.yml",
description: "Potential Ruby On Rails database configuration file",
comment: "Can contain database credentials",
},
SimpleSignature{
part: PartFilename,
match: "omniauth.rb",
description: "OmniAuth configuration file",
comment: "The OmniAuth configuration file can contain client application secrets",
},
SimpleSignature{
part: PartFilename,
match: "settings.py",
description: "Django configuration file",
comment: "Can contain database credentials, cloud storage system credentials, and oth
},
SimpleSignature{
part: PartExtension,
match: ".agilekeychain",
description: "1Password password manager database file",
comment: "Feed it to Hashcat and see if you're lucky",
},
SimpleSignature{
part: PartExtension,
match: ".keychain",
description: "Apple Keychain database file",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".pcap",
description: "Network traffic capture file",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".gnucash",
description: "GnuCash database file",
comment: "",
},
SimpleSignature{
part: PartFilename,
match: "jenkins.plugins.publish_over_ssh.BapSshPublisherPlugin.xml",
description: "Jenkins publish over SSH plugin file",
comment: "",
},
SimpleSignature{
part: PartFilename,
match: "credentials.xml",
description: "Potential Jenkins credentials file",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".kwallet",
description: "KDE Wallet Manager database file",
comment: "",
},
SimpleSignature{
part: PartFilename,
match: "LocalSettings.php",
description: "Potential MediaWiki configuration file",
comment: "",
},
SimpleSignature{
part: PartExtension,
match: ".tblk",
description: "Tunnelblick VPN configuration file",
comment: "",
},
SimpleSignature{
part: PartFilename,
match: "Favorites.plist",
description: "Sequel Pro MySQL database manager bookmark file",
comment: "",
},
SimpleSignature{
part: PartFilename,
match: "configuration.user.xpl",
description: "Little Snitch firewall configuration file",
comment: "Contains traffic rules for applications",
},
SimpleSignature{
part: PartExtension,
match: ".dayone",
description: "Day One journal file",
comment: "Now it's getting creepy...",
},
SimpleSignature{
part: PartFilename,
match: "journal.txt",
description: "Potential jrnl journal file",
comment: "Now it's getting creepy...",
},
SimpleSignature{
part: PartFilename,
match: "knife.rb",
description: "Chef Knife configuration file",
comment: "Can contain references to Chef servers",
},
SimpleSignature{
part: PartFilename,
match: "proftpdpasswd",
description: "cPanel backup ProFTPd credentials file",
comment: "Contains usernames and password hashes for FTP accounts",
},
SimpleSignature{
part: PartFilename,
match: "robomongo.json",
description: "Robomongo MongoDB manager configuration file",
comment: "Can contain credentials for MongoDB databases",
},
SimpleSignature{
part: PartFilename,
match: "filezilla.xml",
description: "FileZilla FTP configuration file",
comment: "Can contain credentials for FTP servers",
},
SimpleSignature{
part: PartFilename,
match: "recentservers.xml",
description: "FileZilla FTP recent servers file",
comment: "Can contain credentials for FTP servers",
},
SimpleSignature{
part: PartFilename,
match: "ventrilo_srv.ini",
description: "Ventrilo server configuration file",
comment: "Can contain passwords",
},
SimpleSignature{
part: PartFilename,
match: "terraform.tfvars",
description: "Terraform variable config file",
comment: "Can contain credentials for terraform providers",
},
SimpleSignature{
part: PartFilename,
match: ".exports",
description: "Shell configuration file",
comment: "Shell configuration files can contain passwords, API keys, hostnames and ot
},
SimpleSignature{
part: PartFilename,
match: ".functions",
description: "Shell configuration file",
comment: "Shell configuration files can contain passwords, API keys, hostnames and ot
},
SimpleSignature{
part: PartFilename,
match: ".extra",
description: "Shell configuration file",
comment: "Shell configuration files can contain passwords, API keys, hostnames and ot
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^.*_rsa$`),
description: "Private SSH key",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^.*_dsa$`),
description: "Private SSH key",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^.*_ed25519$`),
description: "Private SSH key",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^.*_ecdsa$`),
description: "Private SSH key",
comment: "",
},
PatternSignature{
part: PartPath,
match: regexp.MustCompile(`\.?ssh/config$`),
description: "SSH configuration file",
comment: "",
},
PatternSignature{
part: PartExtension,
match: regexp.MustCompile(`^key(pair)?$`),
description: "Potential cryptographic private key",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?(bash_|zsh_|sh_|z)?history$`),
description: "Shell command history file",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?mysql_history$`),
description: "MySQL client command history file",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?psql_history$`),
description: "PostgreSQL client command history file",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?pgpass$`),
description: "PostgreSQL password file",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?irb_history$`),
description: "Ruby IRB console history file",
comment: "",
},
PatternSignature{
part: PartPath,
match: regexp.MustCompile(`\.?purple/accounts\.xml$`),
description: "Pidgin chat client account configuration file",
comment: "",
},
PatternSignature{
part: PartPath,
match: regexp.MustCompile(`\.?xchat2?/servlist_?\.conf$`),
description: "Hexchat/XChat IRC client server list configuration file",
comment: "",
},
PatternSignature{
part: PartPath,
match: regexp.MustCompile(`\.?irssi/config$`),
description: "Irssi IRC client configuration file",
comment: "",
},
PatternSignature{
part: PartPath,
match: regexp.MustCompile(`\.?recon-ng/keys\.db$`),
description: "Recon-ng web reconnaissance framework API key database",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?dbeaver-data-sources.xml$`),
description: "DBeaver SQL database manager configuration file",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?muttrc$`),
description: "Mutt e-mail client configuration file",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?s3cfg$`),
description: "S3cmd configuration file",
comment: "",
},
PatternSignature{
part: PartPath,
match: regexp.MustCompile(`\.?aws/credentials$`),
description: "AWS CLI credentials file",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^sftp-config(\.json)?$`),
description: "SFTP connection configuration file",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?trc$`),
description: "T command-line Twitter client configuration file",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?gitrobrc$`),
description: "Well, this is awkward... Gitrob configuration file",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?(bash|zsh|csh)rc$`),
description: "Shell configuration file",
comment: "Shell configuration files can contain passwords, API keys, hostnames and ot
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?(bash_|zsh_)?profile$`),
description: "Shell profile configuration file",
comment: "Shell configuration files can contain passwords, API keys, hostnames and ot
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?(bash_|zsh_)?aliases$`),
description: "Shell command alias configuration file",
comment: "Shell configuration files can contain passwords, API keys, hostnames and ot
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`config(\.inc)?\.php$`),
description: "PHP configuration file",
comment: "",
},
PatternSignature{
part: PartExtension,
match: regexp.MustCompile(`^key(store|ring)$`),
description: "GNOME Keyring database file",
comment: "",
},
PatternSignature{
part: PartExtension,
match: regexp.MustCompile(`^kdbx?$`),
description: "KeePass password manager database file",
comment: "Feed it to Hashcat and see if you're lucky",
},
PatternSignature{
part: PartExtension,
match: regexp.MustCompile(`^sql(dump)?$`),
description: "SQL dump file",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?htpasswd$`),
description: "Apache htpasswd file",
comment: "",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^(\.|_)?netrc$`),
description: "Configuration file for auto-login process",
comment: "Can contain username and password",
},
PatternSignature{
part: PartPath,
match: regexp.MustCompile(`\.?gem/credentials$`),
description: "Rubygems credentials file",
comment: "Can contain API key for a rubygems.org account",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?tugboat$`),
description: "Tugboat DigitalOcean management tool configuration",
comment: "",
},
PatternSignature{
part: PartPath,
match: regexp.MustCompile(`doctl/config.yaml$`),
description: "DigitalOcean doctl command-line client configuration file",
comment: "Contains DigitalOcean API key and other information",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?git-credentials$`),
description: "git-credential-store helper credentials file",
comment: "",
},
PatternSignature{
part: PartPath,
match: regexp.MustCompile(`config/hub$`),
description: "GitHub Hub command-line client configuration file",
comment: "Can contain GitHub API access token",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?gitconfig$`),
description: "Git configuration file",
comment: "",
},
PatternSignature{
part: PartPath,
match: regexp.MustCompile(`\.?chef/(.*)\.pem$`),
description: "Chef private key",
comment: "Can be used to authenticate against Chef servers",
},
PatternSignature{
part: PartPath,
match: regexp.MustCompile(`etc/shadow$`),
description: "Potential Linux shadow file",
comment: "Contains hashed passwords for system users",
},
PatternSignature{
part: PartPath,
match: regexp.MustCompile(`etc/passwd$`),
description: "Potential Linux passwd file",
comment: "Contains system user information",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?dockercfg$`),
description: "Docker configuration file",
comment: "Can contain credentials for public or private Docker registries",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?npmrc$`),
description: "NPM configuration file",
comment: "Can contain credentials for NPM registries",
},
PatternSignature{
part: PartFilename,
match: regexp.MustCompile(`^\.?env$`),
description: "Environment configuration file",
comment: "",
},
PatternSignature{
part: PartPath,
match: regexp.MustCompile(`credential`),
description: "Contains word: credential",
comment: "",
},
PatternSignature{
part: PartPath,
match: regexp.MustCompile(`password`),
description: "Contains word: password",
comment: "",
},
}
Gitrob 동작 과정 분석
결론
Git repository에 있는 모든 파일을 Scan함.
파일 경로, 파일명, 확장자를 확인.
파일 내용은 확인하지 않음.
자체 Signature를 가지고 해당 Signatrure와 일치하면
Secret 정보를 가지고 있는 파일이라고 판단.
ID, Pass, Key, Token 이외에도 설정 관련 파일은 모두 탐지하는것으로 보임.
1. Scan 하고자 하는 GitHub 주소에 접근할 수 있는 환경변수 설정 후 Gitrob 실행
[Gitrob에게 제공해야하는 개인정보]
GitHub 개인 액세스 토큰
[환경변수]
GITROB_ACCESS_TOKEN
[gitrob 실행 명령어]
./gitrob [git repo id]
2. GitHub 새션 초기화
core.NewSession()
3. GitHub Repo가 몇개 있는지 확인
GatherTargets(sess)
GatherRepositories(sess)
입력된 Github에 repo가 몇개 인지 확인하고 해당 목록을 append함.
4. GitHub Repo 수 만큼 Secret Scan 진행
AnalyzeRepositories(sess *core.Session)
repo 목록에 대한 Secret Scan 시작.
5. GitHub Repo 수 만큼 Thread 생성.
ch = make(chan *core.GithubRepository, len(sess.Repositories))
6. GitHub Repo를 clone 받음
core.CloneRepository(repo.CloneURL, repo.DefaultBranch, *sess.Options.CommitDepth)
git.PlainClone(dir, false, &git.CloneOptions
URL로 저장소를 클론하여 가져옴.
이 때 CloneOptions에 의하여 표준출력으로 진행도가 표시됨.
이미 존재하는 폴더에 클론을 받으면 repository already exists라는 오류가 출력됨.
7. GitHub Repo를 commit history 확인
core.GetRepositoryHistory(clone)
Head returns the reference where HEAD is pointing to.
Log returns the commit history from the given LogOptions.
ForEach call the cb function for each file contained in this iter until an error happens or the end of the iter is reached.
If plumbing.ErrStop is sent the iteration is stop but no error is returned. The iterator is closed.
8. GitHub Repo를 tree 확인
core.GetChanges(commit, clone)
commit.Parents().Next()
commit.Tree()
object.DiffTree(parentCommitTree, commitTree)
CommitObject return a Commit with the given hash. If not found plumbing.ErrObjectNotFound is returned.
Parents return a CommitIter to the parent Commits.
Tree returns the Tree from the commit.
DiffTree compares the content and mode of the blobs found via two tree objects.
9. GitHub Repo Action 확인
core.GetChangeAction(change)
change.Action()
Action returns the kind of action represented by the change, an insertion, a deletion or a modification.
10. GitHub Repo Path 확인
core.GetChangePath(change)
Full path of the node using "/" as separator.
11. GitHub Repo 파일 정보 확인.
core.NewMatchFile(path)
분할은 최종 구분 기호 바로 다음에 경로를 분할하여
디렉토리와 파일 이름 구성 요소로 분리합니다.
경로에 구분 기호가 없으면 Split은 빈 디렉토리와 파일 세트를 경로로 반환합니다.
리턴 된 값에는 path = dir + file 특성이 있습니다.
Ext는 경로에 사용 된 파일 이름 확장자를 반환합니다.
확장자는 경로의 마지막 요소에서 마지막 점에서 시작하는 접미사입니다.
점이 없으면 비어 있습니다.
파일 경로 확인
파일명 확인.
확장자 확인.
12. GitHub Repo 파일 정보 확인.
matchFile.IsSkippable()
확장자, 파일 경로 소문자로 변환,
확장자 : {".jpg", ".jpeg", ".png", ".gif", ".bmp", ".tiff", ".tif", ".psd", ".xcf"}
매칭 시 True
파일 경로 : {"node_modules/", "vendor/bundle", "vendor/cache"}
포함 시 True
이외 : False
True면 아래 단계 진행하지 않고 Pass 처리.
13. GitHub Repo Signatures Scan
signature.Match(matchFile)
정규식을 포함 하고 있는 경우 : patternSignature
정규식과 일치해야하는 경우 : signature
파일 경로 확인
파일명 확인.
확장자 확인.
파일 내용은 확인하지 않음.
참고
반응형
'Tool 사용법 > Git' 카테고리의 다른 글
| Git 코드를 잠깐 저장하고 다른 브랜치로 이동하기 (0) | 2020.09.07 |
|---|---|
| Git add, commit, push 취소하기 (0) | 2020.09.07 |
| Git 원격 저장소에 pull하는 과정 (0) | 2020.09.07 |
| Git 브랜치 방법 (Branch) (0) | 2020.09.07 |
| Git 원격 저장소에 Push 하는 과정 (0) | 2020.09.07 |